Towards an Information Theory of Complex Networks: Statistical Methods and Applications

Matthias Dehmer Frank Emmert-Streib Alexander Mehler Editors Towards an Information Theory of Complex Networks Statist...

Author: Matthias Dehmer | Frank Emmert-Streib | Alexander Mehler

48 downloads 465 Views 6MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Matthias Dehmer Frank Emmert-Streib Alexander Mehler Editors

Towards an Information Theory of Complex Networks Statistical Methods and Applications

Editors Matthias Dehmer UMIT Institute of Bioinformatics and Translational Research Eduard-Wallnöfer-Zentrum I A-6060 Hall in Tirol Austria [email protected]

Frank Emmert-Streib School of Medicine, Dentistry and Biomedical Sciences Center for Cancer Research and Cell Biology Queen’s University Belfast 97 Lisburn Road Belfast BT9 7BL United Kingdom [email protected]

Alexander Mehler Faculty of Computer Science and Mathematics Goethe-University Frankfurt am Main Robert-Mayer-Straße 10 P.O. Box: 154 D-60325 Frankfurt am Main Germany [email protected]

ISBN 978-0-8176-4903-6 e-ISBN 978-0-8176-4904-3 DOI 10.1007/978-0-8176-4904-3 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2011932673 Mathematics Subject Classification (2010): 68R10, 68P30, 94C15 c Springer ScienceCBusiness Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper www.birkhauser-science.com

Preface

For more than a decade, complex network analysis has evolved as a methodological paradigm for a multitude of disciplines, including physics, chemistry, biology, geography, sociology, computer science, statistics, media science, and linguistics. Researchers in these fields share an interest in information processing subject to the networking of their corresponding research object, for instance, genes, molecules, individuals, semes, memes, etc. They start with the insight that any of these research objects is extrinsically characterized, if not constituted, by its networking with objects of the same provenance. In this way, networks, for example, gene networks, food networks, city networks, networks of words, sentences, texts, or web documents become important research objects in more and more disciplines. This book, in line with these research developments, presents theoretical and practical results of statistical models of complex networks in the formal sciences, the natural sciences, and the humanities. One of its goals is to advocate and promote combinations of graph-theoretic, information-theoretic, and statistical methods as a way to better understand and characterize real-world networks. On the one hand, networks appear as paradigmatic objects of approaches throughout the natural and social sciences and the humanities. On the other hand, networks are—irrespective of their disciplinary provenance—known for characteristic distributions of graph-theoretic invariants which affect their robustness and efficiency in information processing. The main goal of this book is to further develop information-theoretic notions and to elaborate statistical models of information processing in such complex networks. In this way, the book includes first steps toward establishing a statistical information theory as a unified basis for complex network analysis across a multitude of scientific disciplines. The book presents work on the statistics of complex networks together with applications of information theory in a range of disciplines such as quantitative biology, quantitative chemistry, quantitative sociology, and quantitative linguistics. It aims to integrate models of invariants of network topologies and dynamic aspects of information processing in these networks or by means of these networks.

v

vi

Preface

Thus, the book is in support of sharing and elaborating models and methods that may help researchers get insights into complex problems emerging from interdisciplinary reasoning. The book is divided into two parts: Chaps. 1–4 deal with formal-theoretical issues of network modeling, while Chaps. 5–13 further develop and apply these methods to empirical networks from a wide range of areas. The book starts with a theoretical contribution by Abbe Mowshowitz on the entropy of digraphs and infinite graphs. The aim is to provide insights into more complex graph models that go beyond the majority of network models based on finite undirected graphs. The chapter by Nicolas Bonichon, Cyril Gavoille, and Nicolas Hanusse presents an informationtheoretic upper bound of planar graphs by means of the newly introduced notion of well-orderly maps. Such a technique might be useful when studying properties of the very important notion of planar graphs. Terence Chan and Raymond W. Yeung study a statistical inference problem using network models. Richard Berkovits, Lukas Jahnke, and Jan W. Kantelhardt examine phase transitions within complex networks that help to examine their structural properties. The remainder of the book combines the theoretical stance of the first section with an empirical analysis of real networks. Elena Konstantinova provides a survey on information-theoretic measures used in chemical graph theory. Prabhat K. Sahu and Shyi-Long Lee develop a model of chemical graphs by example of molecular networks. Exploring the spectral characteristics of these graphs, they provide a successful classification of chemical graphs. Biological or, more specifically, ecological networks are dealt with by Robert E. Ulanowicz who describes a framework of quantifying patterns of the interaction of networked trophic processes from the point of view of information theory. Ecological networks are also the focus of the chapter of Linda J. Moniz, James D. Nichols, Jonathan M. Nichols, Evan G. Cooch, and Louis M. Pecora, who provide an approach to modeling the interaction dynamics of ecosystems and their change. A comprehensive view of ontologically disparate networks is given by Cristian R. Munteanu, J. Dorado, A. Pazos Sierra, F. Prado-Prado, L.G. Pérez-Montoto, S. Vilar, F.M. Ubeira, A. Sanchez-Gonzaléz, M. Cruz-Monteagudo, S. Arrasate, N. Sotomayor, E. Lete, A. Duardo-Sánchez, A. D´ıaz-López, G. Patlewicz, and H. González-D´ıaz who use the notion of entropy centrality to compare various systems such as chemical, biological, crime, and legislative networks, thereby showing the interdisciplinary expressiveness of complex network theory. The book continues with two contributions to linguistic networks: Alexander Mehler develops a framework for analyzing the topology of social ontologies as they evolve within Wikipedia and contrasts them with nonsocial, formal ontologies. Olga Abramov and Tatjana Lokot present a comparative, classificatory study of morphological networks by means of several measures of graph entropy. Edward B. Allen discusses the measurement of the complexity and error probability of software systems represented as hypergraphs. Finally, in the chapter by Philippe Blanchard and Dimitri Volchenkov, random walks are studied as a kind of Markov process on graphs that allow insights into the dynamics of networks as diverse as city and trade and exchange networks.

Preface

vii

With such a broad field, it is clear that the present book addresses an interdisciplinary readership. It does not simply promote transdisciplinary research. Rather, it is about interdisciplinary research that may be the starting point of developing an overarching network science. Matthias Dehmer Frank Emmert-Streib Alexander Mehler

Acknowledgments

Many colleagues have provided us with input, help, and support (consciously or unconsciously) before and during the preparation of this book. In particular, we would like to thank Andreas Albrecht, Gökmen Altay, Gabriel Altmann, Alain Barrat, Igor Bass, David Bialy, Philippe Blanchard, Danail Bonchev, Stefan Borgert, Mieczysław Borowiecki, Andrey A. Dobrynin, Michael Drmota, Ramon Ferrer i Cancho, Maria and Gheorghe Duca, Maria Fonoberova, Armin Graber, Martin Grabner, Peter Gritzmann, Ivan Gutman, Peter Hamilton, Wilfried Imrich, Patrick Johnston, Elena Konstantinova, D. D. Lozovanu, Dennis McCance, Abbe Mowshowitz, Arcady Mushegian, Andrei Perjan, Armindo Salvador, Maximilian Schich, Heinz Georg Schuster, Helmut Schwegler, Andre Ribeiro, Burghard Rieger, Brigitte Senn-Kircher, Fred Sobik, Doru Stefanescu, John Storey, Shailesh Tripathi, Kurt Varmuza, Bohdan Zelinka, and Shu-Dong Zhang. Additionally, Matthias Dehmer thanks Armin Graber for strong support and providing a fruitful atmosphere at UMIT. Finally, we would like to thank our editor Tom Grasso who has been always available and helpful. The work on the chapters of Philippe Blanchard and Dimitri Volchenkov, Olga Abramov, and Alexander Mehler have been supported by the German Federal Ministry of Education and Research (BMBF) through the project Linguistic Networks.1 We gratefully acknowledge this financial support.

1

www.linguistic-networks.net. ix

Contents

1

Entropy of Digraphs and Infinite Networks . . . . . . . . .. . . . . . . . . . . . . . . . . . . . A. Mowshowitz

2

An Information-Theoretic Upper Bound on Planar Graphs Using Well-Orderly Maps . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Nicolas Bonichon, Cyril Gavoille, and Nicolas Hanusse

17

Probabilistic Inference Using Function Factorization and Divergence Minimization . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Terence H. Chan and Raymond W. Yeung

47

3

1

4

Wave Localization on Complex Networks . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Richard Berkovits, Lukas Jahnke, and Jan W. Kantelhardt

75

5

Information-Theoretic Methods in Chemical Graph Theory.. . . . . . . . . Elena Konstantinova

97

6

On the Development and Application of Net-Sign Graph Theory . . . . 127 Prabhat K. Sahu and Shyi-Long Lee

7

The Central Role of Information Theory in Ecology . . . . . . . . . . . . . . . . . . . 153 Robert E. Ulanowicz

8

Inferences About Coupling from Ecological Surveillance Monitoring: Approaches Based on Nonlinear Dynamics and Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 169 L.J. Moniz, J.D. Nichols, J.M. Nichols, E.G. Cooch, and L.M. Pecora

xi

xii

9

Contents

Markov Entropy Centrality: Chemical, Biological, Crime, and Legislative Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 199 C.R. Munteanu, J. Dorado, Alejandro Pazos-Sierra, F. Prado-Prado, L.G. Pérez-Montoto, S. Vilar, F.M. Ubeira, A. Sanchez-Gonzaléz, M. Cruz-Monteagudo, S. Arrasate, N. Sotomayor, E. Lete, A. Duardo-Sánchez, A. D´ıaz-López, G. Patlewicz, and H. González-D´ıaz

10 Social Ontologies as Generalized Nearly Acyclic Directed Graphs: A Quantitative Graph Model of Social Tagging . . . . . . . . . . . . . . 259 Alexander Mehler 11 Typology by Means of Language Networks: Applying Information Theoretic Measures to Morphological Derivation Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 321 Olga Abramov and Tatiana Lokot 12 Information Theory-Based Measurement of Software . . . . . . . . . . . . . . . . . 347 Edward B. Allen 13 Fair and Biased Random Walks on Undirected Graphs and Related Entropies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 365 Philippe Blanchard and Dimitri Volchenkov

Contributors

Olga Abramov University of Bielefeld, Universitätsstraße 25, 33615 Bielefeld, Germany, [email protected] Edward B. Allen Department of Computer Science and Engineering, Mississippi State University, Box 9637, Mississippi State, MS 39762, USA, [email protected] S. Arrasate Department of Organic Chemistry II, Faculty of Science and Technology, University of the Basque Country/Euskal Herriko Unibertsitatea, Apto. 644, 48080 Bilbao, Spain, [email protected] Richard Berkovits Minerva Center and Department of Physics, Bar-Ilan University, Ramat-Gan 52900, Israel, [email protected] Philippe Blanchard Bielefeld – Bonn Stochastic Research Center (BiBoS), University of Bielefeld, Postfach 100131, 33501 Bielefeld, Germany Nicolas Bonichon LaBRI, University of Bordeaux, 351 Cours de la libération, 33405 Bordeaux, France, [email protected] Terence H. Chan Institute for Telecommunications Research, University of South Australia, Adelaide, SA 5095, Australia, [email protected]; [email protected] E.G. Cooch Department of Natural Resources, Cornell University, Ithaca, NY 14853, USA, [email protected] M. Cruz-Monteagudo CEQA, Faculty of Chemistry and Pharmacy, UCLV, Santa Clara 54830, Cuba, [email protected] A. D´ıaz-López Department of Special Public Law, Faculty of Law, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected]

xiii

xiv

Contributors

J. Dorado Department of Information and Communication Technologies, Computer Science Faculty, University of A Coruña, 15071 A Coruña, Spain, [email protected] A. Duardo-Sánchez Department of Special Public Law, Faculty of Law, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected] Cyril Gavoille LaBRI, University of Bordeaux, 351 Cours de la libération, 33405 Bordeaux, France, [email protected] H. González-D´ıaz Department of Microbiology and Parasitology, Faculty of Pharmacy, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected] Nicolas Hanusse LaBRI, CNRS – University of Bordeaux, 351 Cours de la libération, 33405 Bordeaux, France, [email protected] Lukas Jahnke Martin-Luther-Universität Germany

Halle-Wittenberg,

06099

Halle,

Jan W. Kantelhardt Martin-Luther-Universität Halle-Wittenberg, 06099 Halle, Germany Elena Konstantinova Sabolev Institute of Mathematics, Siberian Branch of Russian Academy of Sciences, 630090 Novosibirsk, Russia, e [email protected] Shyi-Long Lee Department of Chemistry and Biochemistry, National Chung Cheng University, Chia-Yi, 621 Taiwan, [email protected] E. Lete Department of Organic Chemistry II, Faculty of Science and Technology, University of the Basque Country/Euskal Herriko Unibertsitatea, Apto. 644, 48080 Bilbao, Spain, [email protected] Tatiana Lokot Faculty of Technology, University of Bielefeld, Universitaetsstr. 25, 33615 Bielefeld, Germany, [email protected] Alexander Mehler Faculty of Computer Science and Mathematics, Goethe University Frankfurt am Main, D-60325 Frankfurt am Main, Germany, [email protected] L.J. Moniz Johns Hopkins University, Applied Physics Laboratory, 11100 Johns Hopkins Road, Laurel, MD 20723, USA, [email protected] Abbe Mowshowitz Department of Computer Science, The City College of New York (CUNY), 138th Street at Convent Avenue, New York, NY 10031, USA, [email protected] C.R. Munteanu Department of Information and Communication Technologies, Computer Science Faculty, University of A Coruña, 15071 A Coruña, Spain, [email protected]

Contributors

xv

J.D. Nichols U.S. Geological Survey, Patuxent Wildlife Research Center, Laurel, MD 20708, USA, [email protected] J.M. Nichols Naval Research Laboratory, Optical Sciences Division, Code 5673, Washington, DC 20375, USA, [email protected] G. Patlewicz Institute for Health and Consumer Protection (IHPC), Joint Research Centre (JRC), European Commission, via E. Fermi 2749–21027 Ispra (Varese), Italy DuPont Haskell Global Centers for Health and Environmental Sciences, Newark, DE 19711, USA, [email protected] Alejandro Pazos-Sierra Department of Information and Communication Technologies, Computer Science Faculty, University of A Coruña, 15071 A Coruña, Spain, [email protected] L.M. Pecora Naval Research Laboratory, Code 6362, Washington, DC 20375, USA, [email protected] L.G. Pérez-Montoto Faculty of Pharmacy, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected] F. Prado-Prado Faculty of Pharmacy, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected] Prabhat K. Sahu Institüt für Physikalische und Theoretische Chemie, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany Department of Chemistry and Biochemistry, National Chung Cheng University, Chia-Yi, 621 Taiwan, [email protected] A. Sanchez-Gonzaléz Department of Inorganic Chemistry, Faculty of Pharmacy, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected] N. Sotomayor Department of Organic Chemistry II, Faculty of Science and Technology, University of the Basque Country/Euskal Herriko Unibertsitatea, Apto. 644, 48080 Bilbao, Spain, [email protected] F.M. Ubeira Department of Microbiology and Parasitology, Faculty of Pharmacy, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected] Robert E. Ulanowicz Department of Biology, University of Florida, Gainesville, FL 32611-8525, USA University of Maryland Center for Environmental Science, Solomons, MD 206880038, USA, [email protected] S. Vilar Faculty of Pharmacy, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain, [email protected]

xvi

Contributors

Dimitri Volchenkov The Center of Excellence Cognitive Interaction Technology (CITEC), University of Bielefeld, Postfach 100131, 33501 Bielefeld, Germany, [email protected] Raymond W. Yeung Department of Information Engineering, The Chinese University of Hong Kong, [email protected]

Chapter 1

Entropy of Digraphs and Infinite Networks A. Mowshowitz

Abstract The information content of a graph G is defined in Mowshowitz (Bull Math Biophys 30:175–204, 1968) as the entropy of a finite probability scheme associated with the vertex partition determined by the automorphism group of G. This provides a quantitative measure of the symmetry structure of a graph that has been applied to problems in such diverse fields as chemistry, biology, sociology, and computer science (Mowshowitz and Mitsou, Entropy, orbits and spectra of graphs, Wiley-VCH, 2009). The measure extends naturally to directed graphs (digraphs) and can be defined for infinite graphs as well (Mowshowitz, Bull Math Biophys 30:225–240, 1968).This chapter focuses on the information content of digraphs and infinite graphs. In particular, the information content of digraph products and recursively defined infinite graphs is examined. Keywords Digraphs • Networks

•

Entropy

•

Infinite graphs

•

Information content

MSC2000 Primary 68R10; Secondary 05C20, 05C25, 05C75, 94C15, 90B10.

1 Introduction 1.1 Overview This chapter investigates the information content of directed and infinite graphs. The information content of a finite graph (directed or undirected) is a quantitative measure based on the symmetry structure of the graph. As explained in detail A. Mowshowitz () Department of Computer Science, The City College of New York (CUNY), 138th Street at Convent Avenue, New York, NY 10031, USA e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 1, © Springer Science+Business Media, LLC 2011

1

2

A. Mowshowitz

below, the group of symmetries of a finite graph partitions the vertex set and thus induces a unique finite probability scheme. The entropy of this scheme is taken to be the information content of the graph. This “classical” notion differs from “graph entropy” introduced in [16]. Development of the concept of entropy applied to finite graphs is discussed in [17] and [20]. The application of entropy to graphs was introduced in the 1950s soon after the appearance of Shannon’s famous paper on information theory. Entropy measurement has been used as a tool for characterizing molecules and chemical structures. For example, measures characterizing the structural complexity of chemical graphs have been developed and applied in [1, 3, 6]. Most of these measures are based on graph invariants that generate an equivalence relation on the vertices or edges of a graph. The resulting equivalence classes form a partition to which a finite probability scheme [14] can be associated in a natural way. The entropy of such a scheme provides a quantitative measure of structural complexity. Various structural features of a graph have provided the basis for entropy measures. The earliest centered on the symmetries of a graph [21]. Other features, such as branching structure in molecular graphs, have been used to define entropy measures [8]. Measures associated with graphs representing atoms and molecules have been defined and applied to problems of discriminating chemical isomers and to classifying atomic and chemical structures [7, 9, 15]. Such measures have also been used for the analysis of biological networks [13]. Degree characteristics of a graph have been used as basis for an entropy-based measure of disorder in complex networks [23]. Interest in measuring the information content of graphs has also been kindled in recent years by the growing importance of computer and social networks in modern society [10, 24]. Relationships between graph entropy-based measures, expressed as inequalities, have been demonstrated in [11]. The notion of information content can be extended to infinite graphs. The approach adopted here is to consider an infinite graph as a sequence of finite graphs. Each of the finite graphs in the sequence has a well-defined information content, and if the corresponding sequence of information content values has an unambiguous limit, that limit is defined to be the information content of the given infinite graph. In Sect. 2, we will look into the existence of directed graphs with prescribed information content and determine the information content of certain products of directed graphs. Section 3 will focus on infinite graphs, investigating the information content of some special classes of infinite graphs, and applying results from Sect. 2 to determine the information content of infinite graphs in general. Section 4 will examine some applications of the information measure to problems in network theory.

1.2 General Definitions Definition 1. G D .V; E/; jV j < 1; E V2 is called a finite undirected graph. If G D .V; E/; jV j < 1, and E V V , then G is called a finite directed graph.

1 Entropy of Digraphs and Infinite Networks

3

Definition 2. A digraph Ln D .V; E/ is called a (directed) path of length n ( 1), if V D fv0 ; v1 ; ; vn g and E D f.v0 ; v1 /; .v1 ; v2 /; ; .vn1 ; vn /g. The number of vertices in Ln is n C 1, one more than the number of edges. Definition 3. A digraph Cn D .V; E/ is called a (directed) cycle of length n ( 2), if V D fv1 ; v2 ; ; vn g and E D f.v1 ; v2 /; .v2 ; v3 /; ; .vn ; v1 /g. Cn has the same number (n) of vertices and edges. Definition 4. The complete graph Kn has n vertices and n2 (undirected) edges. See [12] for additional definitions of basic concepts in graph theory.

2 Entropy of Digraphs 2.1 Definition and Examples The automorphism group of a digraph and the measure of information content based on the group are defined below. Definition 5. Let G D .V; E/ be a (directed or undirected) graph with vertex set V (with jV j D n), and edge set E. The automorphism group of G, denoted by Aut.G/, is the set of all adjacency preserving bijections of V . Definition 6. Let fVi j1 i kg be the collection of orbits of Aut.G/ and suppose jVi j D ni f or 1 i k. The entropy or information content of G is given by the following formula [17]: Ia .G/ D

k X ni i D1

n

log

n i

n

:

Figure 1.1 illustrates the computation of the information content of a digraph.

2.2 Entropy of Digraph Products Many different binary operations on graphs and digraphs appear in the literature [19]. We will examine four such operations in some detail, namely, the sum, join, Cartesian product, and the composition. Our aim is to determine the information content of a digraph operation in relation to the information contents of the respective digraphs in the operation. Such products are useful in defining classes of digraphs with properties of interest in different applications, especially those pertaining to the analysis of networks. Definition 7. The sum of G1 and G2 is the digraph G1 [ G2 defined by V .G1 [ G2 / D V .G1 / [ V .G2 / and E.G1 [ G2 / D E.G1 / [ E.G2 /.

4

A. Mowshowitz 1

7

2

1

2

6

6

3

1

2

3

3

5

5

4

4 Y

X

Z Orbits: {1}, {2}, {3} Ia (Z) = log 3

Orbits: {1}, {2,5}, {3,6}, {4,7} Ia (Y) = −(1/7) log (1/7) −3(2/7) log (2/7)

Orbit: {1,2,3,4,5,6} Ia (X) = 0

Fig. 1.1 Computation of information content

G

H

G

U

H

G+H

GXH

GoH

Fig. 1.2 Binary operations on digraphs

Definition 8. The join of G1 and G2 is the digraph G1 C G2 defined by V .G1 C G2 / D V .G1 /[V .G2 / and E.G1 CG2 / D E.G1 /[E.G2 /[fŒu; vju 2 V .G1 /; v 2 V .G2 /g where Œu; v denotes the undirected edge joining u and v. Definition 9. The Cartesian product of G1 and G2 is the digraph G1 G2 given by V .G1 G2 / D V .G1 / V .G2 / and E.G1 G2 / D f.u; v/ D ..u1 ; u2 /; .v1 ; v2 //ju1 ; v1 2 V .G1 /; u2 ; v2 2 V .G2 /, and either u1 D v1 and .u2 ; v2 / 2 E.G2 / or u2 D v2 and .u1 ; v1 / 2 E.G1 /g Definition 10. Two digraphs G and H are relatively prime with respect to the Cartesian product if whenever G is isomorphic to G 0 D and H is isomorphic to H 0 D, then D is the identity digraph K1 . Definition 11. The composition of G1 and G2 is the digraph G1 ı G2 given by V .G1 ı G2 / D V .G1 / V .G2 / and E.G1 ı G2 / D f.u; v/ D ..u1 ; u2 /; .v1 ; v2 //ju1 ; v1 2 V .G1 /; u2 ; v2 2 V .G2 /, and either .u1 ; v1 / 2 E.G1 / or u1 D v1 and .u2 ; v2 / 2 E.G2 /g The foregoing operations are illustrated in Fig. 1.2.


5

These binary operations will be discussed with a view to characterizing the information content of digraphs resulting from their application.

2.3 Sum and Join Theorem 1. Let G and H be digraphs. (a) Suppose Aut.G/ has orbits ViG with jViG j D mi for 1 i m, and Aut.H / has orbits ViH with jViH j D ni for 1 i n. If no component of G is isomorphic to a component of H , then Ia .G [ H / D Ia .G C H / D log.n C m/ C

1 ŒnIa .G/ C mIa .H / nCm n log.n/ m log.m/:

(b) If G and H are isomorphic, then Ia .G [ H / D Ia .G C H / D Ia .G/. More generally, if each Gi .1 i n/ is isomorphic to G, then Ia .G1 [ G2 [ Gn / D Ia .G1 C G2 C Gn / D Ia .G/: Proof. Ia .G [ H / D Ia .G C H / since the orbits of Aut.G [ H / are the same as those of Aut.G C H /. This is a consequence of the fact that every vertex of G is adjacent to every vertex of H in G C H . (a) Ia .G [ H / D Ia .G C H / D k k k k P P P P ni mi ni mi 1 log. nCm / log. nCm / D nCm Œ ni log.nCm/C mi log.nC nCm nCm i D1

m/ C

k P

1 Œ ni nCm i D1

i D1

log.ni /

k P i D1

i D1

mi log.mi / D log.n C m/ C

mIa .H / n log.n/ m log.m/, as required. (b) See [18].

i D1

1 ŒnIa .G/ nCm

C t u

When the two digraphs are of equal size, the information content of their join is just one more than their average information content. Corollary 1. Let G and H be as in the Theorem. If n D m, then Ia .G [ H / D Ia .G C H / D 12 ŒIa .G/ C Ia .H / C 1. Proof. The result follows immediately from the Theorem by setting m D n in the expression for Ia .G [ H / D Ia .G C H /. t u

6

A. Mowshowitz Y x

Z

Fig. 1.3 Partial joins

Corollary 2. Let G and H be as in the Theorem and suppose Ia .G/ D Ia .H /. 1 Then (i) Ia .G [ H / D Ia .G C H / D Ia .G/ C log.n C m/ nCm Œn log.n/ C m log.m/, and (ii) if in addition n D m, Ia .G [ H / D Ia .G C H / D Ia .G/ D Ia .G/ C 1. Of particular importance to the representation of real network growth is the partial join operation. Definition 12. A partial join of G1 and G2 for the set F is the digraph G1 ˚ G2 defined by V .G1 ˚G2 / D V .G1 /[V .G2 / and E.G1 ˚G2 / D E.G1 /[E.G2 /[F , where F fŒu; vju 2 V .G1 /; v 2 V .G2 /g. Figure 1.3 illustrates partial join operations for different sets F . The information content of a partial join depends on the set F . For example, if both graphs G and H are isomorphic to the directed cycle of length n and G ˚ H is defined for set F consisting of a single undirected edge, Ia .G ˚ H / D log.n/ since each orbit of Aut.G ˚ H / consists of two of the 2n vertices. If there are two edges joining G and H , one of which does not join corresponding vertices of the directed n-cycles, the information content is log.2n/ since Aut.G ˚H / is the trivial group in this case. Note that Ia .G/ D Ia .H / D 0 since the automorphism group of a directed cycle with n vertices is the cyclic group of order n. Thus, it appears that Ia .G ˚ H / can be expressed in terms of Ia .G/ and Ia .H / in special cases only. Theorem 2. Let G1 and G2 be complete graphs with m and n vertices, respectively, and suppose G D G1 ˚ G2 is a partial join with jF j D 1. m1 m1 m1 n1 log mCn log mCn (a) If m is different from n, Ia .G/ D mCn C mCn C 2 log.m C n/ mCn (b) If m D n, then Ia .G/ D

n i 1h .n 1/ log C log.n/ : n n1

Proof. Let Œx; y be the edge in F where x is in G1 and y is in G2 . If m is different from n, the partial join G has four orbits A, B, C , and D, where A consists of the m 1 vertices of G1 excluding x, B consists of the n 1 vertices of G2 excluding y, and C and D are singletons containing x and y, respectively. If m D n there are two orbits with 2 and 2.n 1/ vertices, respectively. t u


7

2.4 Cartesian Product and Composition Theorem 3 ([18]). (a) Ia .G H / Ia.G/CIa.H / for any digraphs G and H . (b) Equality holds when G and H are weakly connected and relatively prime with respect to the Cartesian product. Proof. Part (a) follows from the fact that Aut.G H / is a subgroup Aut.G/ Aut.H /. Part (b) is a consequence of the fact that Aut.G H / is isomorphic to Aut.G/ Aut.H / if and only if digraphs G and H are relatively prime with respect to the Cartesian product. Note that being relatively prime is a sufficient but not a necessary condition for equality in the theorem. t u The information content measure is also sub-additive for the composition operation. Theorem 4 ([18]). Ia .G H / Ia.G/ C Ia.H / for any digraphs G and H . Figure 1.4 provides examples of the information content of the Cartesian product and composition.

2.5 Existence Theorem The join and Cartesian product can be used to construct digraphs with given information content. More precisely, for any finite probability scheme there exists a digraph with information content equal to the entropy of the scheme. This result is stated in the following theorem originally presented in [18]. Theorem 5. Let n be any positive integer, and suppose P D fnij g is a partition of n where nij D ni (1 j ri /, ni1 ¤ ni2 (i1 ¤ i2 ), and i D 1; 2; ; k. Then there exists a weakly connected digraph G with n vertices such that Aut.G/ has exactly k P rD ri orbits, and for each nij there is an orbit A with jAj D nij ; and, hence, i D1

Ia .G/ D H.P / D

k X i D1

ri

n ni i : log n n

Proof. The proof is based on a simple construction. Let Gi D Lri 1 Cni where Lri 1 is a directed path of length ri 1 and Cni is a directed cycle of length ni . Since the path and cycle are relatively prime with respect to the Cartesian product, the orbits of Aut.Gi / are the respective products of the orbits of Aut.Lri 1 / and Aut.Cni /. Hence, Aut.Gi / has exactly ri orbits, each consisting of ni elements. The digraph G formed by taking the join of the k non-isomorphic Gi has an automorphism group with orbits corresponding to the partition specified in the hypothesis of the theorem, and thus has the required information content. t u Figure 1.5 illustrates the Theorem for n D 25; P D f13 ; 24 ; 32 ; 42 g.

8

A. Mowshowitz

G H Ia (G) = 0; Ia (H) = log 3

G’ H’ Ia (G’) = Ia (H’) = log 3 − 2/3

GXH

G’ X H’

HoG

Ia (G’ X H’) = 2 log 3 − 16/9 < Ia (G’) + Ia (H’) = 2 log 3 − 12/9

Ia (G o H) = Ia (H) + Ia (G) = log 3

Ia (G X H) = Ia (G) + Ia(H) = log 3

G Ia (G) = 0;

H Ia (H) = log 3;

Fig. 1.4 Information content of Cartesian product and composition

+

L2 X C1

+

+

L3 X C2

L1 X C3

Fig. 1.5 Construction of digraph with prescribed information content

L1 X C4


9

3 Entropy of Infinite Graphs 3.1 Preliminaries Definition 13. A graph G D .V; E/ is countable if jV [ Ej is countable. G is locally finite if the degree of every vertex of G is finite. In what follows, we will restrict attention to countable graphs that may or may not be locally finite. Definition 14 ([18]). Let G D .V; E/ be a countable graph. A sequence fGn g1 nD1 of finite graphs Gn with Vn D V .Gn / and En D E.Gn / is said to converge to G as a limit (written limn!1 Gn D G) if limn!1 Vn D V .G/ and limn!1 En D E.G/. Note that both V and E are simply the limits of sequences of sets. Definition 15 ([18]). A sequence fGn g1 nD1 of finite graphs Gn is a defining sequence for a countable graph G if Gn GnC1 for every n, and limn!1 Gn D G. Since the limit of any monotonically increasing sequence fAn g1 nD1 of sets An exists 1 S and is equal to An , every countable graph G has a defining sequence. nD1

A defining sequence for a countable graph G with V .G/ D fv1 ; v2 ; v3 ; g can be constructed as follows: V .G1 D fv1 g and E.G1 / D ;, V .GnC1 / D V .Gn / [ fvnC1g and E.GnC1 / D E.Gn / [ fŒvnC1; u 2 E.G/ju 2 V .Gn /g. Definition 16 ([18]). Let fGn g1 nD1 be a defining sequence for a countable graph G. The information content IO.GI Gn / of G with respect to the sequence fGn g1 nD1 is given by IO.GI Gn // D limn!1 Ia .Gn / if the limit exists. Figure 1.6 shows a countable graph with defining sequences that give rise to different information content values.

,

G1

,

,

G3

G2

... , G4

Fig. 1.6 A countable graph with more than one defining sequence

10

A. Mowshowitz

ia .Gn / D

n

0 if n is odd log.5/ 35 log.3/ 25 if n is even

Thus, for the subsequence Sn consisting

of the odd terms, IO.GI Sn / D 0; and for the subsequence Tn consisting of the even terms, IO.GI Tn / D log.5/ 35 log.3/ 25 . The difference in this case is finite, but it could be infinite as shown in [18]. Using a measure that depends on the graph’s defining sequence is not necessarily a disadvantage. An infinite graph can be viewed as an idealization of a growth process. Including the defining sequence in the definition allows for capturing different principles of growth in practice.

3.2 Classes of Infinite Graphs Infinite graphs can be built up recursively with the aid of graph products. The following result makes use of the Cartesian product. Lemma 1. Let G be a graph with n vertices. Ia .G K2 / D Ia .G/. Proof. Corresponding vertices of the two copies of G are in the same orbit of G K2 , so G and G K2 have the same number of orbits, and each orbit of G K2 has exactly double the number of vertices as the corresponding orbit of G. Thus, if Aut.G/ has orbits A1 ; A2 ; ; Ar with jA.i /j D ki ; 1 i r, then Ia .G K2 / D r P 2ki i log 2k t u D Ia G. 2n 2n i D1

Suppose G is a graph with n vertices. If Aut.G/ is the identity group, then Ia .G/ D log.n/, and Ia .G K2 / D log.n/. The sequence H1 D G; HnC1 D Hn K2 ; for n 1 serves as a defining sequence of an infinite graph. Since Ia .Hn / D log.n/, limn!1 Ia .Hn / D 1. At the other extreme is the hypercube Hn , which can be defined recursively as follows: H1 D K2 ; HnC1 D Hn K2 ; for n 1: Since the limit of the (defining) sequence fHn g1 nD1 exists, we can set H1 D limn!1 Hn . Now, Ia .Hn / D 0 for all n 1 which implies by the lemma that IO.H I Hn / = 0, i.e., the sequence of finite hypercubes yields a limit whose information content is zero. The hypercube serves as a useful model in parallel computation. A key feature in this context is the favorable maximum distance between any two vertices in the graph. This allows for placing computational units so as to minimize communication costs. The zero information content of the


11

W3 X K2

S3 X K2

Fig. 1.7 Star and wheel products

hypercube reflects the high degree of symmetry of this graph, which allows for simultaneous placement of elements at optimal distance from each other. Other graphs of interest, with information content between the two extremes, can be substituted for G in G K2 . Let S k denote the star of order k, a connected graph with one vertex of degree k 1 and k 1 vertices of degree 1; and let W k denote the wheel of order k, a connected graph obtained from the star by joining the degree 1 vertices in a cycle of length k1. Once again using the Cartesian product, we can build infinite sequences based on these simple graphs. S1k D S k ; k D Snk K2 ; for n 1: SnC1

A sequence of graphs Wnk can be defined k1 similarly. Ia .S k / D Ia .W k / D k1 log log.k 1/. k1 log k1 D log.k/ k1 k k k Denoting by S1 and W1 , respectively, the infinite graphs with defining sequences k1 k 1 k k O O fSnk g1 nD1 and fWn gnD1 , we have I .S1 I Sn / D I .W1 I Wn / D log.k/ k log.k 1/. As k increases, almost all the vertices fall into one orbit and the information content tends to zero. Figure 1.7 shows the Cartesian products, respectively, of the star and the wheel with K2 . The information content of the line graph of order k is given by: ( k

Ia .L / D

log

k

k1 k

2

log

k

2

C

1 k

log k

if n is even if n is odd:

The information content of the line graph increases without bound, so the information content of the limit graph is infinite. The cycle graph of order k has information content Ia .C k / D 0, so the limit graph in this case has information content zero.

12

A. Mowshowitz

More complex graphs could be constructed by substituting for K2 in the Cartesian products defining the terms in the infinite sequences considered above.

4 Applications Preferential attachment has been studied extensively as a protocol for the growth of large-scale networks like the Internet [5]. According to this protocol, a vertex added to a network will be more likely to become attached to existing vertices of higher rather than of lower degree. The “preference” of a vertex v as a target of attachment might be expressed as the probability given by the degree of v divided by the sum of the degrees in the graph. This introduces a random element in the growth process. Perhaps the simplest way to realize a (relatively deterministic) version of growth by preferential attachment is to add a single new vertex at each iteration, connecting the new vertex to an existing one whose degree is maximal in the current graph. Call this a type-0 preferential attachment protocol. If the starting graph is K1 , the result is clearly a star. After the nth new vertex has been added, a star of order n C 1 has n been formed. This graph S nC1 has information content log.n C 1/ nC1 log.n/, and as noted above, this value tends to zero as n increases without bound. A variation on this simple protocol is to add k new vertices at each iteration and attach each one of them to a different existing vertex, choosing the existing vertices in nonincreasing order of degree, beginning with one of maximal degree. Figure 1.8 illustrates the construction process according to this protocol, and the following theorem gives the information content in the case where k equals the number of vertices in the initial graph of the sequence. Theorem 6. Let fGnk g1 1 be a sequence of graphs defined as follows: G1k D S k k GnC1 is obtained from Gnk by adding k new vertices and joining each one to a different vertex of maximal degree in Gnk .

G41 = S5

G42

G43

Fig. 1.8 A graph constructed with preferential attachment protocol type-0


H21 = K2

H22

13

H23

Fig. 1.9 A graph constructed with preferential attachment protocol type-1

The information content of Gnk is given by: Ia .Gnk /

.k 1/.n 1/ k nk C 1 nk C 1 D C log log nk C 1 .k 1/.n 1/ nk C 1 k 1 nk C 1 n1 C log log.nk C 1/: C nk C 1 n1 nk C 1

Proof. Since k vertices are added for each iteration, Gnk , the nth graph in the sequence has nk C 1 vertices. Let v be the vertex of highest degree in Gnk . The orbits of Aut.Gnk / consist of the vertex v alone, the vertices of degree 1 adjacent to v, the vertices of degree > 1 adjacent to v, and the vertices of degree 1 at distance 2 from v. Thus, the orbits of Aut.Gnk / have 1, n 1, k, and .k 1/.n 1/ vertices from which the result follows. t u Corollary 3. Let Gnk be defined as in the theorem. Then IO.GI Gnk / D log k. Proof. Simplifying the expression in the theorem gives Ia .Gnk / D log.nk C 1/ 1 Œ.k 1/.n 1/ log.k 1/.n 1/ C .n 1/ log.n 1/ C k log k. Taking the nkC1 t u limit as n ! 1 yields IO.GI Gnk / D log k as required. Greater connectivity in a network that grows by preferential attachment can be achieved by allowing the newly added vertices to be joined to more than one existing vertex [2]. Call this a type-1 preferential attachment protocol. This protocol is illustrated in Fig. 1.9. The following theorem gives the information content of an infinite graph that grows according to a type-1 protocol with k D 2. Theorem 7. Let fHn2 g1 1 be a sequence of graphs defined as follows: H12 D K2 2 HnC1 is obtained from Hn2 by adding 2 new vertices and joining each one to exactly two different vertex of maximal degree in Hn2 .

14

A. Mowshowitz

(a) The information content of Hn2 is given by: Ia .Hn2 /

n 1 nC1 nC1 D C : log log nC1 n nC1 1

(b) IO.H I Hn2 / D 0 Proof. Every new vertex is joined to the two vertices of K2 ; so Aut.Hn2 / has exactly two orbits consisting of 2 and 2n vertices, respectively. Part (b) follows immediately n 1 from the fact that nC1 and nC1 tend to 1; and nC1 log.n C 1/ tends to 0. t u n More realistic protocols call for making estimates of information content. If, for example, each of the k new vertices added at a given step was joined to an existing vertex v with probability equal to degree(v)/sum of degrees in the current graph, one would try to obtain upper and lower bounds on the information content of the infinite graph.

5 Summary and Conclusion In this chapter, we have investigated the information content of directed graphs and infinite graphs. We have determined the information content of various graph products, computed the information content of some special classes of infinite graphs, and investigated the information content of infinite graphs whose growth is governed by preferential attachment protocols. Digraphs and infinite graphs offer useful tools for modeling networks. In particular, the information content of digraphs and infinite graphs may be useful for classifying networks according to critical features such as diameter and vulnerability. The relationship between information content and these features is an open question. For example, the graphs in Fig. 1.10 differ by the maximum possible on the

1

1

6

2

6

2

5

3

5

3 4

4 G

Fig. 1.10 Information content and diameter

Ia (G) = log 6 diameter G = 3

H Ia (H) = 0 diameter H = 3


15

measure of information content, but both have diameter 3 and for both the minimum number of vertices whose removal disconnects the respective graphs is 2. This example suggests that there is no simple correlation between information content, on one hand, and diameter and vulnerability, on the other hand. Two graphs with the same diameter can register as far apart as possible on the information content scale; similarly, two graphs with the same (vertex) vulnerability rating can be far apart on the information content measure. However, it is possible that graphs that grow in a certain way (like the hypercube) may exhibit a high correlation between diameter and information content. Information content could possibly be correlated with diameter if only graphs with similar distance properties were grouped together. In the example, the maximum distances from each vertex (from 1 to 6) to any other in G is given by .2; 2; 3; 2; 2; 3/, whereas for H these maximum distances are .3; 3; 3; 3; 3; 3/. Although the two graphs have the same diameter, the sequences of maximum distance do not coincide. Another potentially useful discriminant is the metrical property distance degree sequence [4]. This sequence gives the number of vertices at each succeeding distance from a given vertex. Graph H in Fig. 1.10 is distance degree regular but G is not. The hypercube is distance degree regular and has 0 information content; however, a distance degree regular graph does not necessarily have a transitive automorphism group. The more general metrical properties investigated in [22] might prove useful in determining the relationship between information content and diameter. Another open question is how to define the information content of an infinite graph independently of a defining sequence of finite graphs. Investigation of this question could shed light on the properties of very large-scale networks. Acknowledgments Research was sponsored by the US Army Research Laboratory and the UK Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US Army Research Laboratory, the US Government, the UK Ministry of Defence, or the UK Government. The US and UK Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.

References 1. Balaban, A.T., Balaban, T.S.: New vertex invariants and topological indices of chemical graphs based on information on distances. J. Math. Chem. 8, 383–397 (1991) 2. Bent, G., Dantressangle, P., Vyvyan, D., Mowshowitz, A., Mitsou, V.: A dynamic distributed federated database. Proceedings of the Second Annual Conference of the International Technology Alliance, Imperial College, London, September 2008 3. Bertz, S.H.: The first general index of molecular complexity. J. Am. Chem. Soc. 103, 3241–3243 (1981) 4. Bloom, G., Kennedy, J.W., Quintas, L.V.: Distance degree regular graphs. In: Chartrand, G. (ed.) The Theory of Applications of Graphs, pp. 95–108. Wiley, New York (1981)

16

A. Mowshowitz

5. Bollobás, B., Riordan, O.: The diameter of a scalefree random graph. Combinatorika 24, 5–34 (2004) 6. Bonchev, D.: Information Theoretic Indices for Characterization of Chemical Structures. Research Studies Press, Chichester, UK (1983) 7. Bonchev, D.: Complexity in Chemistry, Biology, and Ecology. Mathematical and Computational Chemistry series. Springer, New York (2005) 8. Bonchev, D., Trinajstic, N.: Information theory, distance matrix and molecular branching. J. Chem. Phys. 67, 4517–4533 (1977) 9. Dehmer, M., Emmert-Streib, F.: Structural information content of chemical networks. Zeitschrift für Naturforschung A 63a, 155–159 (2008) 10. Dehmer, M., Emmert-Streib, F., Mehler, A. (eds.): Towards an Information Theory of Complex Networks: Statistical Methods and Applications. Springer/Birkhäuser, Berlin (2011) 11. Dehmer, M., Mowshowitz, A.: Inequalities for entropy-based measures of network information content. Appl. Math. Comput. 215, 4263–4271 (2010) 12. Harary, F.: Graph Theory. Addison Wesley, Reading, MA (1969) 13. Hirata, H., Ulanowicz, R.E.: Information theoretical analysis of ecological networks. Int. J. Syst. Sci. 15, 261–270 (1984) 14. Khinchin, A.I.: Mathematical Foundations of Information Theory. Dover Publications, New York (1957) 15. Konstantinova, E.V., Skorobogatov, V.A., Vidyuk, M.V.: Applications of information theory in chemical graph theory. Indian J. Chem. 42, 1227–1240 (2002) 16. Körner, J.: Coding of an information source having ambiguous alphabet and the entropy of graphs. In: Transactions of the 6th Prague Conference on Information Theory, 411–425 (1973) 17. Mowshowitz, A.: Entropy and the complexity of graphs: I. An index of the relative complexity of a graph. Bull. Math. Biophys. 30, 175–204 (1968) 18. Mowshowitz, A.: Entropy and the complexity of graphs: II. The information content of digraphs and infinite graphs. Bull. Math. Biophys. 30, 225–240 (1968) 19. Mowshowitz, A., Mitsou, V., Bent, G.: Models of network growth by combination. Proceedings of the Second Annual Conference of the International Technology Alliance, Imperial College, London, September 2008 20. Mowshowitz, A., Mitsou, V.: Entropy, orbits and spectra of graphs. In: Dehmer, M. (ed.) Analysis of Complex Networks: From Biology to Linguistics. Wiley-VCH, Weinheim (2009, in press) 21. Rashevsky, N.: Life, information theory, and topology. Bull. Math. Biophys. 17, 229–235 (1955) 22. Skorobogatov, V.A., Dobrynin, A.A.: Metric analysis of graphs. MATCH 23, 105–151 (1988) 23. Sole, R.V., Valverde, S.: Information theory of complex networks: On evolution and architectural constraints. Lect. Notes Phys. 650, 189–207 (2004) 24. Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications, Structural Analysis in the Social Sciences. Cambridge University Press, Cambridge (1994)

Chapter 2

An Information-Theoretic Upper Bound on Planar Graphs Using Well-Orderly Maps Nicolas Bonichon, Cyril Gavoille, and Nicolas Hanusse

Abstract This chapter deals with compressed coding of graphs. We focus on planar graphs, a widely studied class of graphs. A planar graph is a graph that admits an embedding in the plane without edge crossings. Planar maps (class of embeddings of a planar graph) are easier to study than planar graphs, but as a planar graph may admit an exponential number of maps, they give little information on graphs. In order to give an information-theoretic upper bound on planar graphs, we introduce a definition of a quasi-canonical embedding for planar graphs: well-orderly maps. This appears to be an useful tool to study and encode planar graphs. We present upper bounds on the number of unlabeled1 planar graphs and on the number of edges in a random planar graph. We also present an algorithm to compute wellorderly maps and implying an efficient coding of planar graphs. Keywords Compact coding • Enumerative combinatorics • Planar embedding • Planar graph MSC2000 Primary 05C10; Secondary 05C10, 05C30, 05C85.

1 Introduction In graph theory, a planar graph is a graph which can be embedded in the plane, i.e., it can be drawn on the plane in such a way that its edges intersect only at their endpoints. A planar graph drawn in the plane without edge intersections is called a

1

Nodes and edges are not assumed to be labeled.

N. Bonichon () LaBRI, University of Bordeaux, 351 Cours de la libération, 33405 Bordeaux, France e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 2, © Springer Science+Business Media, LLC 2011

17

18

N. Bonichon et al.

planar map or a planar embedding of the graph. The class of planar graphs is one of the most studied graphs. How much information can contain a simple planar graph of n nodes? The question is highly related to the number of planar graphs. Counting the number of (non-isomorphic) planar graphs with n nodes is a well-known and long-standing unsolved graph-enumeration problem (cf. [24]). There is no known close formula, neither asymptotic nor even an asymptotic on the logarithm of this number. Any asymptotic on the logarithm would give a bound on the number of independent random bits needed to generate a planar graph uniformly at random (but not necessary in polynomial time). Random combinatorial object generation is an important activity regarding average case complexity analysis of algorithms and testing algorithms on typical instances. Unlike random graphs (the Erdös–Rényi graph Model), still little is known about random planar graphs. Indeed adding an edge in a planar graph highly depends on the location of all previous edges. Random planar maps, i.e., plane embeddings of planar graphs, have been investigated more successfully. Schaeffer [35] and then Banderier et al. [2] have showed how to generate in polynomial time several planar map families, e.g., 3-connected planar maps. Unfortunately, this generating does not give much information about random planar graphs because there are many ways to embed a planar graph into the plane. On the positive side, some families of planar graphs support efficient random generation: trees [1], maximal outerplanar graphs [3, 14], and more recently labeled and unlabeled outerplanar graphs [4]. Besides the combinatorial aspect and random generation, an important attention is given in Computer Science to efficiently represent discrete objects. Efficiently means that the representation is succinct, i.e., the storage of these objects uses few bits, and that the time to compute such representation is polynomial in their size. Fast manipulation of the so-encoded objects and easy access to a part of the code are also desirable properties. At least two scopes of applications of high interests are concerned with planar graph representation: Computer Graphics and Networking. Surface discretization of a 3D object outputs a list of 3D coordinates and a set of adjacency relations. In the case of convex objects, the set of adjacency relations is an unlabeled planar graph. In general, small degree faces are used for surface discretization, with triangle or quad meshes. Then, a compressor is applied on the planar graph. Performances are expressed averaging the number of bits per edge or per node. They are evaluated among a benchmark of standard examples [21], due to the lack of “good” random planar graph generator, or typical instance generator. For example, King and Rossignac [22, 34] gave a triangulation compressor that guarantees 3:67 bits per node, the best possible rate being log2 .256=27/ 3:24 bits per node from Tutte’s enumerative formula [39]. Routing table design for a network has been investigated in the case of planar networks [15, 16, 26, 37]. The underlying graph of the network is preprocessed to optimize routing tables, a data structure dedicated to each node in charge of finding the next output port given the destination address of an incoming message. The main objective is to minimize the size of the routing tables while maintaining routes as short as possible. The strategy used by Gavoille and Hanusse [16] based on a

2 An Information-Theoretic Upper Bound on Planar Graphs

19

k-page embedding, and then improved by Lu [26] with orderly spanning trees, demonstrates that a compact planar graph representation helps for the design of compact routing tables, especially when shortest paths are required.

1.1 Related Works Succinct representation of n-node m-edge planar graphs has a long history. Turán [38] pioneered a 4m bit encoding, which has been improved later by Keeler and Westbrook [20] to 3:58m. Munro and Raman [29] then proposed a 2m C 8n bit encoding based on the 4-page embedding of planar graphs (see [40]). In a series of articles, Lu et al. [8,11] refined the coding to 4m=3 C5n, thanks to orderly spanning trees, a generalization of Schnyder’s trees [36]. Independently, codings have been proposed for triangulations, where m D 3n6. A 4n bit encoding has been obtained by several authors [5,11,34], interestingly with rather different techniques, and then improved by the Rossignac’s Edgebreaker [22], who guaranteed 3:67n bits for triangulations and computable in O.n/ time. Actually, He et al. [19] showed that, in O.n log n/ time, a space optimal encoding for triangulations and for unlabeled planar graphs can be achieved. Hence, a O.n log n/ time and a 3:24n bit encoder for triangulations exist. For that, they use a recursive separator decomposition of the graph, and an exponential coding algorithm for the very end components of sublogarithmic size. However, the time complexity hidden in the big-O notation could be of limited use in practice. To implement the encoder, one needs, for instance, to implement planar isomorphism and Lipton–Tarjan planar separator [23]. The time complexity has been recently improved to O.n/ for planar graphs by Lu [27]. Although the length of the coding is optimal, the approach of [19, 27] does not give any explicit bound of the number of bits used in the representation. If we are interested only in the information-theoretic bound of planar graphs or in statistical properties of planar graphs (what a random planar graph looks like: number of edges, connectivity, etc.), other tools can be used. Denise et al. [12] specified a Markov chain on the space of all labeled planar graphs whose limit distribution is the uniform distribution. Their experiments show that random planar graphs have approximately 2n edges, and are connected but not 2-connected. Although the Markov chain converges to the uniform distribution, it is not proved whether this Markov chain becomes close enough to the uniform distribution after a polynomial number of steps. It is, however, proved that almost all labeled planar graphs have at least 1:5n edges, and that the number p.n/ of unlabeled planar graphs satisfies that n1 log2 p.n/ tends to a constant such that log2 .256=27/ 6 6 log2 .256=27/ C 3. The bounds on easily derive from Tutte’s formula [39]: Triangulations are planar graphs, and every planar graph is a subgraph of a triangulation, thus having 23n6 possible subsets of edges. There are also no more than nŠ2 nCo.n/ labeled planar graphs as there are at most nŠ ways to label the nodes of a graph.

20

N. Bonichon et al.

Osthus et al. [31] investigated triangulations containing any planar graph, and they showed that there is no more than nŠ25:22nCo.n/ labeled planar graphs. Osthus et al. [31] also showed that almost all labeled planar graphs have at most 2:56n edges, and that almost all unlabeled planar graphs have at most 2:69n edges. A lower bound of 13n=7 1:85n has been obtained by Gerke and McDiarmid [17], improving the 1:5n lower bound of the expected number of edges of [12]. Properties of random planar graphs have also been investigated in [28]. Gimenez and Noy [18] show that the number of edges of a random labeled planar graph is asymptotically normal and the mean is 2:213n and variance is 0:4303n. Unlike general graphs, labeled and unlabeled planar graphs do not have the same growing rate (up to the nŠ term) as proved in [28]. So upper bounds on labeled planar graphs do not transfer to upper bounds on unlabeled planar graphs, but the reverse is true. Using generating function techniques, Noy and Gimenez [18] proved that the number of labeled connected planar graphs tends to nŠ24:767nCO.log n/ . The number of simple planar maps is asymptotic to 25:098nCO.log n/ (cf. algebraic generating function presented in [25]) providing an upper bound for unlabeled planar graphs.

1.2 Presented Results In this chapter, we present a new representation of planar graphs called well-orderly maps. Starting from a planar graph, we show how to build and encode a well-orderly map in linear time. Our construction leads to counting results about planar graphs. More precisely, we show an upper bound of 24:91nCo.n/ on p.n/, the number of unlabeled planar graphs with n nodes. Since our upper bound can be parameterized with the number of edges, and using the lower bound of [18], we are able to show that almost all unlabeled graphs have at least 1:85n edges and at most 2:44n edges, setting a new lower bound and improving the 2:69n upper bound of [31]. The presented results are a synthesis of results presented in [6, 7].

1.3 Outline of the Chapter Let us sketch our technique. Since the number of useful combinatorial objects are numerous, we first briefly describe in Fig. 2.1 the different steps toward the compact coding of planar graphs. Our real starting point is a planar map, sometimes called planar embedding. To get a planar map from a planar graph, well-known linear time algorithm can be used (see for instance [9]). Roughly speaking, we first present a very particular embedding of a planar graph called well-orderly map and show how to encode it using combinatorial tools like bijective combinatorics and a specific compression technic of binary strings.


21

Planar graph 6 compressed binary strings Planar map 5+1=6 binary strings

Well−orderly map

Super−triangulation = Minimal Realizer = 3 well−orderly trees GRAPH THEORY AND GRAPH ALGORITHMIC

Balanced trees with n−2 inner vertices

COMBINATORICS AND BINARY TEXT COMPRESSION

Fig. 2.1 Roadmap toward the compact coding of a planar graph

A First Upper Bound Based on Triangulation A natural approach to represent an n-node planar graph G is to consider a triangulation of G, i.e., a supergraph S of G such that S is planar, has n nodes and 3n 6 edges. Then, G can be obtained by coding S and a set MS of edges such that E.S / n MS D E.G/. This way of representing a planar graph is suggested by the .log2 .256=27/ C 3/n D 6:24n bit upper bound of [12] mentioned above. Introduction of Well-Orderly Maps and Super-Triangulation To obtain a representation more compact than 6:24n bits, we need to carefully construct S . In particular, crucial steps are the way we embed G into the plane, and the way we triangulate its faces. In Sect. 2.2, we introduce a specific embedding of G called a well-orderly map, and we show that it can be computed in linear time. Given a well-orderly map, we present how to build the supergraph S of G, called hereafter super-triangulation and defined in Sect. 2. More precisely, a supertriangulation S has the property that for a given node v 2 S only, one can perform in a unique manner a traversal of S by following a specific spanning tree T rooted in v, called a well-orderly tree, such that T is contained in G. Hence, given the supertriangulation S of G, MS is of cardinality at most .3n 6/ .n 1/ D 2n 5, and the edges of MS can be described among the possible edges of S n T only, i.e., with at most 2n bits. This already provides a .log2 .256=27/ C 2/n D 5:24n bit upper bound. Observe that the case G not connected can be easily transformed (in linear Q e.g., by linking all the connected components time) into a new connected graph G, of G into a single node (see Sect. 4 for more details). Using Minimal Realizer Properties The next step consists in encoding in a very compact way the super-triangulation. In Sect. 3.1, we represent the super-triangulation S by a realizer, that is a partition

22

N. Bonichon et al.

of the edges into three trees .T0 ; T1 ; T2 / (see Schnyder’s trees [36]). In our case, the partition has specific properties and corresponds to minimal realizer. We also show how to uniquely recover the three trees from such a super-triangulation. Different properties of minimal realizer are useful since the knowledge of two well-orderly trees implies a canonical description of the third one and can exploited to save bits. At this point, the following properties are only given as an illustration: • Every edge .u; v/ of S such that (1) u is the parent of v in T1 and (2) u is an inner node in T2 , must belong to G. This significantly saves bits in the coding of MS since many edges of G can be guessed from S . • An extra property is that two nodes belonging to the same branch of T2 have the same parent in T1 (a branch is a maximal set of related nodes obtained in a clockwise depth-first search of the tree, and such that a node belongs to only one branch at the time, see Sect. 3). This latter property simplifies a lot the representation of S . Knowing T2 , T1 does not need to be fully represented. Only one relevant edge per branch of T2 is enough. As any tree of a realizer can be deduced from the two others, the representation of S can be compacted in a very efficient way, storing for instance T2 and the relevant edges of T1 . Combining such properties and the optimal coding of realizer using the bijection of Poulalhon and Schaeffer [33] (see also Theorem 2), we get an encoding of supertriangulation presented in [7]. Compact Coding in Binary Strings Finally, we show in Sect. 3 that the explicit representation of G is done with six binary strings of different density (namely the ratio between the number of “ones” it contains and its length): five for S and one for MS . We compact each string with a variant of the Pagh’s compressor [32]. This allows to reach an optimal entropy coding, i.e., with log2 kn Co.n/ bits for an n-bit string of k ones.2 If we parametrize the number of branches of T2 (or equivalently its number of leaves), an entropy analysis shows that 4:91n bits are enough to represent G.

2 Embedding and Triangulating Algorithms A plane embedding of a graph, or shortly a plane graph, is a mapping of each node to a point of the plane and of each edge to the continuous curve joining the two ends of this edge such that edges do not cross except, possibly, on a common extremity. A graph that has a plane embedding is a planar graph.

2

The original compressor runs in expected linear time. We give in this chapter a simpler guaranteed linear time construction with asymptotically the same performances.

2 An Information-Theoretic Upper Bound on Planar Graphs Fig. 2.2 Relationship between realizer and orderly tree: (a) edge-orientation rule around a node for a realizer, and (b) blocks ordering around an orderly node (T is represented by directed edges because the edge .v; w/ of T1 )

23

a

b

T2

BC

T1 B

BP

In this chapter, we deal with simple (no loops and no multi-edges) and undirected graphs. If we cut the plane along the edges, the remainder falls into connected regions of the plane, called faces. Each plane graph has a unique unbounded face, called the outerface. The boundary of a face is the set of incident edges. The interior edges are the edges non-incident to the boundary of the outerface, similarly for interior nodes. Precise definitions can be founded for instance in [13, 30]. A triangulation is a plane embedding of a maximal planar graph, that is a planar graph with n nodes and 3n 6 edges. There is only one way to embed in the plane (up to a continuous transformation), a maximal planar graph whose three nodes are chosen to lie on the outerface.

2.1 Well-Orderly Tree, Realizer and Super-Triangulation Let T be a rooted spanning tree of a plane graph H . Two nodes are unrelated if neither of them is an ancestor of the other in T . An edge of H is unrelated if its endpoints are unrelated. We introduce well-orderly trees, a special case of orderly spanning trees of Chiang, Lin, and Lu in [8], referred as simply orderly trees later. Let v1 ; : : : ; vn be the clockwise preordering of the nodes in T (nodes ordered by their first visit in a clockwise traversal of the tree T ). Recall that a node vi is orderly in H with respect to T if the incident edges of vi in H form the following four blocks (possibly empty set of vertices) in clockwise order around vi (see Fig. 2.2b): • • • •

BP .vi /: the edge incident to the parent of vi B< .vi /: unrelated edges incident to nodes vj with j < i BC .vi /: edges incident to the children of vi B> .vi /: unrelated edges incident to nodes vj with j > i

A node vi is well orderly in H with respect to T if it is orderly, and if: • The clockwise first edge .vi ; vj / 2 B> .vi /, if it exists, verifies that the parent of vj is an ancestor of vi (in T ). In other words, if .vi ; vj / the first edge of B> .vi /, then the parent of vj is an ancestor of vi in T .

24

N. Bonichon et al. r1

a

r2

w

u

b

r2

r1

w

u

t

t

v

v

r0

r0

Fig. 2.3 Two realizers for a triangulation. The tree T 0 rooted in r0 (the tree with bold edges augmented with the edges .r0 ; r1 / and .r0 ; r2 /) is well orderly in (b), and simply orderly in (a) (the node v is not well orderly: .v; w/ is the clockwise first edge of B> .v/ and the parent of t is not an ancestor of v). The clockwise preordering of T 0 in (a) is r0 ; r2 ; v; u; t; w; r1

Definition 1 (well-orderly tree). T is a well-orderly tree of H if all the nodes of T are well orderly in H , and if the root of T belongs to the boundary of the outerface of H (similarly for simply orderly tree). Note that an orderly tree (simply or well orderly) is necessarily a spanning tree. Observe also that the incident edges in H of a node of T are either in T or unrelated. In particular, if an edge of H is related (i.e., one endpoint is a descendant of the other one in T ), then it has to belong to T . It follows that all the neighbors in H of the root of T are in T . Definition 2 (well-orderly map). A plane graph H is a well-orderly map rooted in v if H has a well-orderly tree of root v. A convenient way to manipulate triangulations is to deal with realizers. Definition 3 (realizer). A realizer of a triangulation is a partition of its interior edges in three sets T0 , T1 , T2 of directed edges such that for each interior node v it holds (see Fig. 2.2a): • The clockwise order of the edges incident with v is: leaving in T0 , entering in T1 , leaving in T2 , entering in T0 , leaving in T1 , and entering in T2 . • There is exactly one leaving edge incident with v in T0 , T1 , and T2 . Observe that if .T0 ; T1 ; T2 / is a realizer, then .T1 ; T2 ; T0 / and .T2 ; T0 ; T1 / are also realizers. Cyclic permutations of a realizer are not in general the only distinct realizers of a given triangulation. Figure 2.3 depicts two realizers for a same


25

triangulation. Actually, the number of n-node realizers is asymptotically 24nCO.log n/ (cf. [5]), whereas the number of triangulations is only .256=27/nCO.log n/ (cf. [39]). Schnyder showed in [36] that each set Ti of a realizer induces a tree rooted in one node of the outerface. Moreover, he described a linear time algorithm to compute such trees. Hereafter, if R D .T0 ; T1 ; T2 / is a realizer, then for notational convenience R also denotes the underlying triangulation. There are strong relationships between realizers and orderly trees (see Fig. 2.2). In every realizer R D .T0 ; T1 ; T2 /, T0 (and by cyclic permutation each Ti ) is an orderly tree of R n fr1 ; r2 g, where ri denotes the root of Ti . Indeed, the incident edges with any node v that are not in T0 (thus that are unrelated with T0 ) are either clockwise before the entering edges of T0 or clockwise after. Conversely, let T be an orderly tree of a triangulation. Observe that the root of T has at least two children (because its root is of degree at least two and all its neighbors must be in T ), and thus T has at least two leaves. A realizer .T0 ; T1 ; T2 / can be obtained from T setting T0 D T n fr1 ; r2 g, where r1 ; r2 are, respectively, the clockwise last and first leaf of T – actually it is not difficult to see that ri is the root of Ti – and setting, for all inner nodes v, that the clockwise first edge of B> .v/ and the clockwise last edge of B< .v/ belong to T1 and T2 , respectively (as illustrated in Fig. 2.2b). Observe that this latter assignment for T1 and T2 is the only possible realizer with T0 D T n fr1 ; r2 g. For each tree Ti of a realizer, we denote by T i the tree composed of Ti augmented with the two edges of the outerface incident to the root of Ti . A node of a rooted tree is inner if it is neither the root nor a leaf. For every non-root node u 2 Ti , we denote by pi .u/ the parent of u in Ti . Definition 4 (super-triangulation). A realizer S D .T0 ; T1 ; T2 / is a supertriangulation of a graph G if: 1. 2. 3. 4.

V .S / D V .G/ and E.G/ E.S / E.T0 / E.G/ T 0 is a well-orderly tree of S For every inner node v of T2 , .v; p1 .v// 2 E.G/

Intuitively, a super-triangulation of a graph G is a specific triangulation of the faces of a specific plane embedding of G. Before exploring more deeply the properties of super-triangulations, observe that, from Definition 4, the tree T0 does not span in general the graph G (cf. example in Fig. 2.4). Moreover, a non-connected graph may have a super-triangulation. For example, if G has an edge and two isolated nodes, in that case, E.T0 / D E.G/ is possible. Theorem 1. Every connected planar graph with at least three nodes has a supertriangulation, computable in linear time. In particular, Theorem 1 implies that every connected planar graph has an embedding which is a well-orderly map.

26

N. Bonichon et al.

b

v6

v5

a

v5

v2

v2

v8

v8

v4 v7

v6

v1

v4

v7

v3

v3

c v6

v1 v5 v8

v2

v4 v7 v3 v1

Fig. 2.4 A planar graph G (a), a well-orderly map of G rooted at v1 with its well-orderly tree (bold edges) (b), and a super-triangulation of G (c) (dotted edges are non-edges of G)

2.2 Computing a Super-Triangulation from a Well-Orderly Map In order to prove Theorem 1, we need the next three lemmas. The proofs of these lemmas are given after the proof of Theorem 1. Lemma 1. Every well-orderly map rooted in some node v has a unique well-orderly tree of root v. Lemma 2. Let G be a connected planar graph, and let v be any node of G. Then G has a well-orderly map of root v. Moreover, well-orderly trees and the well-orderly map can be computed in linear time. In [8], a result similar to Lemma 2 about simply orderly trees and embeddings is proved. However, the extra condition reduces much more the choice of the embedding for the input planar graph and leads to the uniqueness of the tree


27

(Lemma 1). In the case of simply orderly embeddings, several orderly trees may exist (cf. Fig. 2.3 where both orderly trees T 0 span the same triangulation). Actually, the uniqueness concerns also the way to triangulate the faces of well-orderly maps, thanks to the next lemma. Lemma 3. Let T be the well-orderly tree of H rooted in some node r0 , and assume that T has at least two leaves. Let r2 and r1 be the clockwise first and last leaves of T , respectively. Then, there is a unique super-triangulation .T0 ; T1 ; T2 / of the underlying graph of H , preserving the embedding H , and such that each Ti has root ri . Moreover, T0 D T n fr1 ; r2 g and the super-triangulation are computable in linear time. First of all, let us show that Lemmas 1, 2, and 3 imply Theorem 1. Proof of Theorem 1. Consider a connected planar graph G with at least three nodes, and let v be any node of G with the only constraint that if G is a path, then v is chosen to be of degree two (this is feasible since G has at least three nodes). Thanks to Lemma 2, one can compute in linear time a well-orderly map H of G and a wellorderly tree T rooted in v. Let us show that T has at least two leaves r1 ; r2 lying on the outerface of H , r2 traversed before r1 in a clockwise preordering of T . We show that T cannot be a chain, and thus has a node with at least two children (and thus has two leaves). If G is a path, then T rooted in a node of degree two is not a chain. Assume that G is not a path, but T is a chain. Then there exists an edge of G that is not in T . However, all pairs of nodes of a chain are related, thus must belong to T . Therefore, T is not a chain. Lemma 3 can be therefore applied, and one can compute for G a supertriangulation in linear time. t u Proof of Lemma 1. Assume that H has two well-orderly trees T; T 0 rooted in v. Let v1 ; : : : ; vn (resp. v01 ; : : : ; v0n ) be the clockwise preordering of the nodes of T (resp. T 0 ). Let vi be the node such that the neighbors of vi in T and in T 0 differ, and such that i is minimum. We have vt D v0t for all t 6 i , and BC .vi / ¤ BC0 .vi /, where BC0 .vi / denotes the children edge block around vi in T 0 . W.l.o.g. assume jBC .vi /j 6 jBC0 .vi /j (the symmetric case is proved by exchanging the role of T and T 0 ). Note that B< .vi / D B> .vi / D ¿ is impossible, otherwise BC .vi / would consist of all the neighbors of vi (maybe the vi ’s parent excepted) and jBC .vi /j 6 jBC0 .vi /j and BC .vi / ¤ BC .vi / would be incompatible. Let e1 (resp. e2 ) be the clockwise first (resp. last) edge of BC .vi /. Let e be an arbitrary edge of BC0 .vi /. In the following, e1 6 e means either e1 D e, or e1 is clockwise before e around vi . Let us show that e1 6 e. This is clearly true if B< .vi / D ¿. If B< .vi / ¤ ¿, then consider any edge .vi ; vh / 2 B< .vi /. Then, .vi ; vh / … BC0 .vi /. Indeed, as h < i , the path from vh to vi in T exists also in T 0 , and the edge .vi ; vh / of T 0 would create a cycle in T 0 . Thus, e1 6 e.

28

N. Bonichon et al.

If B> .vi / D ¿, then e 6 e2 . Hence, e1 6 e 6 e2 which is incompatible with the fact that BC0 .vi / and BC .vi / are blocks of consecutive edges such that jBC .vi /j 6 jBC0 .vi /j. Thus, we must have B> .vi / ¤ ¿. Let .vi ; vj / be the clockwise first edge of B> .vi /. Then, .vi ; vj / … BC0 .vi /. Indeed, as T is well orderly, the vj ’s parent in T , say vk , is an ancestor of vi , so k < i . As BC .vk / D BC0 .k/ for k < i , the edge .vk ; vj / exists in T 0 . Hence, the path from vj to vi in T exists also in T 0 , and the edge .vi ; vj / of T 0 would create a cycle in T 0 . It follows that every edge e 2 BC0 .vi / is such that .vi ; vj / 6 e and e ¤ .vi ; vj /. As the path from vi to vj in T exists also in T 0 , the node vj is after vi in a clockwise preorder of T 0 . It follows that vi is not well orderly in T 0 : a contradiction. t u Proof of Lemma 2. We first give a simple algorithm to construct a well-orderly map of G. Then we give some hints for an O.n/ time implementation. We start by computing an arbitrary plane embedding H of G such that v belongs to the outerface of H . This can be done in O.n/ time [10]. Then we traverse H from v to build a well-orderly tree T . However, not every plane embedding allows the construction of a well-orderly tree. If during the construction, T does not span all the nodes, the embedding of H is modified, and a new traversal is run again. We show that, after a finite number of steps, the number of nodes covered by the traversal increase, and so the construction converges to a well-orderly tree. To describe more precisely the traversal and the modification of H , we need some definitions. Let T be any tree of H rooted in v, not necessarily a spanning tree. A node is free if it does not belong to T . An edge is free if one of its endpoints if free. We extended the notion of well-orderly node as follows: a node is partially well orderly (with respect to H and T ) if it is well orderly except that its edge-blocks B< and B> (relative to a clockwise preordering of the nodes of T ) may contain zero or more free edges. Moreover, the clockwise last edge of B< and the clockwise first edge of B> are never free. T is partially well orderly if all the nodes of T are partially orderly. Every well-orderly tree is a partially well-orderly tree that spans H . The four edge-blocks around a partially well-orderly node u in T are denoted by BP .v; T /, B< .v; T /, BC .v; T /, and B> .v; T /. For convenience, the clockwise last edge of B< .v; T / is named the back-edge of u in T , and the clockwise first edge of B> .v; T / is named the front-edge in T . We run a procedure traversal .H; v/ applied on the current embedding H of G, which returns a partially well-orderly tree T of H rooted in v (see Fig. 2.5 for example). It initializes T WD fvg and treats v as follows: (1) List in a clockwise manner around v the edges .v; u1 /; : : : ; .v; uk / that are after the back-edge and before the front-edge of v in T (if the back-edge or the frontedge does not exist, the list consists of all the edges incident with v not already in T ) (2) Update T with the edges .v; u1 /; : : : ; .v; uk / (3) Recursively treat the nodes u1 ; : : : ; uk in that order

2 An Information-Theoretic Upper Bound on Planar Graphs Fig. 2.5 The partial tree constructed by traversal .H; v/ is in bold. Free nodes are drawn white. The dotted edge is the new location of e after running flip .H; e; .vi ; u/; .vj ; w//

29

S

vk e

vi+1

w C

u

vi

vm

vj

vl v

Let v1 ; : : : ; vp be the clockwise preordering of the nodes of T , the tree returned by traversal .H; v/. Consider a node vi , and let Tvi be the tree obtained by traversal .H; v/ after the treatment of vi . A key observation is that B< .vi ; Tvi / D B< .vi ; T / and that B> .vi ; Tvi / D B> .vi ; T /. In particular, the back-edge and frontedge of vi with respect to T and with respect to Tvi (if they exist) are the same. After the treatment of vi , the edges around vi in Tvi form the four (possibly empty) edge-blocks BP .vi ; Tvi /, B< .vi ; Tvi /, BC .vi ; Tvi /, and B> .vi ; Tvi /. Hence in T , the edge-blocks around vi are: BP .vi ; T /, B< .vi ; T /, BC .vi ; T /, and B> .vi ; T /. To show that T is partially well orderly, it remains to show that if .vi ; vj / 2 B> .vi ; T / is the front-edge, then the parent of vj in T is an ancestor of vi . When visiting the node vi in T , the edges of the tree constructed up to vi (i.e., Tvi 1 ) are either between nodes vt with t < i , or are .vk ; vj / with k < i and j > i . Moreover, vk belongs to the path from vi to the root of T , v. Thus a front-edge .vi ; vj / is such that the parent of vj is an ancestor of vi . Therefore, T is partially well orderly. Assume that T does not cover all the nodes (if T is a spanning tree, then we are done). Let vi be a node of T having an incident free edge. W.l.o.g., we assume that B> .vi ; T / contains a free edge (cf. Fig. 2.5). The case where vi has a free edge in B< .vi ; T / is symmetric. Let ei D .vi ; u/ be the clockwise last free edge of B> .vi ; T /. (Actually one can choose any free edge that clockwise ends a block of free edges in B> .vi ; T /). By definition, B> .vi ; T / contains at least one unrelated edge (in particular the front-edge). Hence, let e D .vi ; vj / be the clockwise last unrelated edge of B> .vi ; T / that is before ei . Finally, let ej D .vj ; w/ be the clockwise first free edge of vj before e and such that there is no unrelated edge between ej and e (so ej is the first edge of the block of free edges just before e). If such edge does not exist, we set ej WD e. In other words, e, ei , and ej are chosen such that the edges between e and ei in vi , and between e and ej in vj form a maximal block of free edges. We change the embedding H by running the procedure flip .H; e; ei ; ej / that works as follows: In vi , e is moved and inserted clockwise after ei , and in vj , e is moved and inserted clockwise before ej . For convenience, we say that a flip has been performed around e.

30

N. Bonichon et al.

Once H has been updated by flip .H; e; ei ; ej /, we reapply traversal on the new embedding. Procedures traversal and flip are so applied up to get a spanning tree. To complete the correctness, we need to show that flip keeps planarity, and that the partial well-orderly tree obtained after calling traversal converges to a spanning tree. Let X be the set of free nodes forming the free edges between e and ei , and between e and ej . For every x 2 X , let Cx denote the connected component containing x in the subgraph of H induced by the free nodes. Let C D [x2X Cx . To prove that flip.H; e; ei ; ej / keeps planarity, we show that every path P from y 2 C to the root v contains either vi or vj . Let R be the bounded connected region of R2 n B, where B is the cycle composed of the path in T (the tree obtained before calling flip) from vi to vj , and closed by the edge .vi ; vj /. Assume that P contains a node vk 2 T with vk 2 R [ B, and k … fi; j g. W.l.o.g., assume that vk is the first node T from y in P , and let .vk ; z/ be the free edge of P . We have .vk ; z/ 2 B< .vk ; T /, or .vk ; z/ 2 B> .vk ; T /. Let us assume .vk ; z/ 2 B< .vk ; T /, the other case is symmetric. As B< .vk ; T / ¤ ¿, vk has a back-edge. The back-edge is .vk ; vi /. Indeed, if the back-edge is .vk ; vt /, t ¤ i , then the cycle composed of the ˚ path in T between vk and vt , and closed by .vk ; vt /, would disconnect Cz and vi ; vj , and vi or vj has a free neighbor in Cz . Thus the edge .vi ; vk / exists and is clockwise after .vi ; vj / since it belongs to R. As .vi ; vk / disconnects vj from Cz , there must exist a free edge .vi ; s/, for some s 2 Cz . This edge is clockwise after .vi ; vk /: a contradiction with the definition of .vi ; vj /. Therefore, the embedding returned by flip .H; e; ei ; ej / is a plane embedding of G. Observe that whenever e has been moved below C , thanks to flip, the tree returned by a new call to traversal contains T . Indeed, C is connected to GnC only by vi and vj . Thus, the move of e cannot create an unrelated edge .vt ; x/ with x 2 C and t … fi; j g. Assume that after moving e, traversal does not visit any new node. Then, either vi has a front-edge and a free edge clockwise after its front-edge, or vj has a back-edge and a free edge clockwise before its back-edge. Indeed, if not, all the neighbors of vi and of vj would be related, and the size of T would increase. Assume that vi has front-edge e 0 and a free edge. It follows that a next call to flip can be applied on an edge e 0 clockwise before e. Hence, in at most deg.vi / calls to flip, a new free neighbor of vi (or of vj ) will be visited. Observe that for every directed edge e, there is at most one flip around e. On Fig. 2.5, after running flip around e, and calling traversal, the tree T is augmented with (at least) the edge .vj ; w/. This completes the correctness of the well-orderly map algorithm for G. Let us evaluate its time complexity. There is at most O.n/ calls to traversal and to flip (as there is at most one call to flip per directed edges), each one taking O.n/ time. Thus, a naive implementation of that algorithm gives a O.n2 / time algorithm. We first remark that flip can be implemented in O.1/ worst-case time, using double pointers for the incident edge list of a node, and using for each edge a pointer for each endpoint toward the edge pointer in the incident list. Moreover, as the tree grows by adding edges, the construction of the whole tree costs O.n/ time. The only difficulty is to efficiently manage the edges e, ei , and ej for preparing the call to flip.


a

31

b

12

10

6

11

2 3

5 7

9

8 4 1

Fig. 2.6 An application of the recursive algorithm with the visit ordering of nodes (a), and the final embedding with the well-orderly tree (b). Node labeled i is visited at step i , that is the edge of the tree between i and its parent is created at step i . When node 6 is reached, a recursive step is run on the nodes 3, 4, and 5, below the edge .2; 6/. In node 3, after a trivial recursive step (treatment of node 4), a flip around .3; 5/ is performed. This allows to visit the nodes 7 and 8. At the time node 5 is treated, a flip around .5; 7/, and then around .5; 2/ is performed, allowing to visit the nodes 9, 10, and 11, completing the recursive step below the edge .2; 6/. Then, the last node 12 is visited

Note that when one treats a node vi with a front-edge .vi ; vk /, we have choice to continue the construction of T (the notations refer to the example depicted on Fig. 2.5): either one can continue T from vi , by treating vi C1 , or we can consider the subgraph S delimited by the cycle composed of the path from vi to vk in T and close by .vi ; vk /, and one can recursively apply the treatments of the nodes of S (by treating vm , vl , and vj in Fig. 2.5). If in Tvi the nodes of S have been visited in the order vi1 ; : : : ; vir , then the nodes are recursively treated in the order vir ; : : : ; vi1 . Indeed both parts of the embedding (the part after vi , and the part inside S ) cannot interact because of the edge e. The part of the tree composed of the nodes after vj can be computed after computing the trees for S and after applying flip .H; e; ei ; ej /. It is not difficult to see that the recursive version of the algorithm P allows to manage e, ei , and ej with a total of O. niD1 deg.vi // D O.n/ time. Figure 2.6 shows an example of the recursive algorithm. t u Proof of Lemma 3. Let T be the well-orderly tree of H rooted in r0 , and let T0 D T n fr1 ; r2 g, where r2 ; r1 are the clockwise first and last leaves of T . We first show that r1 and r2 belong to the boundary of the outerface of H . Consider Pi the path in T from r0 to ri , for i 2 f1; 2g. All the nodes of Pi must belong to the outerface, in particular ri . Indeed, by induction (this is true for r0 ), a node v of P2 (resp. P1 ) has an empty edge-bock B< .v/ (resp. B> .v/). Thus, the clockwise first (resp. last) children of v (if it exists) must belong to the outerface. Let H 0 (resp. G 0 ) be the plane graph (resp. planar graph) obtained from H (resp. from G) and augmented with the three edges between the ri ’s – keeping planarity as r1 ; r2 lie on the outerface – such that they form the boundary of the outerface of H 0 . (Each edge is added only if it does not create multi-edges.) As the edges

32

N. Bonichon et al.

between the ri ’s belong to any super-triangulation of G, it remains to show that the super-triangulation S D .T0 ; T1 ; T2 / for G 0 preserving H 0 and with ri root of Ti , is unique. The super-triangulation for G must be same, S (these three edges cannot create inner node for T2 ). We will first show how to construct S and then show that it is unique. First observe that T0 is a tree rooted in r0 (removing two leaves from T maintains its connectivity). Clearly, E.T0 / E.G/ E.G 0 /, and T 0 is a well-orderly tree of H 0 rooted in r0 . By Lemma 1, T 0 is unique. Let us construct the set T1 . Let vi … fr0 ; r1 g be a node of T 0 . We apply the following assignment rules (for the induction, we assume that the assignment has been applied to r0 D v1 ; : : : ; vi 1 , and that p1 .r0 / D r1 ): 1. The clockwise first edge of B> .vi / (this edge-block is relative to T 0 ), if it exists, belongs to T1 . 2. If B> .vi / D ¿, then the edge leaving vi to the child of p0 .vi / immediately after vi clockwise around p0 .vi /, if it exists, belongs to T1 . 3. If B> .vi / D ¿, and if vi is the clockwise last child of p0 .vi /, then .vi ; p1 .p0 .vi /// belongs to T1 . Let us check that H 0 [T1 is still a plane graph. No edge is added if Rule 1 applies. For Rule 2, there are no edges incident to p0 .vi / between vi and its next sibling, keeping planarity of the embedding. And for Rule 3, p0 .vi / has no children between vi and p1 .vi /, and allows to freely connect vi with p1 .vi / since, by induction, one can assume that the property holds for every node vj , j < i . Let us check that fT0 ; T1 g are two sets of a realizer. We have seen that T 0 is well orderly. Every non-root node vi has assigned a parent in T1 . Hence, T1 is connected. We check that in each of the three rules, the parent of vi in T1 is assigned to a node vj of T 0 with j > i . As a consequence, T1 has no cycle and is connected, so it is a tree. We also check that the edge between vi and its parent in T1 is clockwise after the children (if they exist) of vi in T0 , and clockwise before the edge to p0 .vi /. Hence, such T1 set is compatible with the edge-orientation rule of realizers. At this step H 0 [ T1 may contain edges that are assigned neither to T 0 nor to T 1 . Let X D E.H 0 / n .E.T 0 / [ E.T 1 // be this edge set. Constructing T2 can be done using Property 6. As there is only one way to select T2 from fT0 ; T1 g, we need to check that the edges of X are compatible with such set T2 . Let e be an arbitrary edge of X . Assume e D .vi ; vj / with i < j . Since e … T0 , then e 2 B> .vi /. Moreover, e … T1 implies that e is not the clockwise first edge of B> .vi /. Indeed, the clockwise first edge of B> .vi / has been assigned to T1 by Rule 1. Therefore, e 2 T2 fulfills the edge-orientation rule of realizers. It follows that S D .T0 ; T1 ; T2 / is a realizer of the plane graph H 0 [ T1 [ T2 . We have seen that E.T0 / E.G/ E.G 0 / and that T 0 is well orderly in H 0 . By the assignment rules, we remark that if the edge .v; p1.v// … E.G 0 / (Rule 2 or 3), then v cannot have any child in T2 (the edge .v; p1 .v// forms a triangle with some T0 and T1 edges). In other words, for every inner node v of T2 , .v; p1 .v// 2 E.G 0 / (actually .v; p1 .v// 2 E.G/). Thus, S is a super-triangulation for G 0 .


33

It remains to show that S is the unique super-triangulation of G 0 that preserve H and with ri root of Ti . As T0 is unique, and as, given fT0 ; T1 g, T2 is unique, it remains to prove that T1 is unique. The clockwise first edge of B> .vi / must belong to T1 since the parent of vi in T1 must be clockwise before the edges of T2 entering in vi . If B> .vi / D ¿, and if vi has a sibling vj immediately clockwise after vi around p0 .vi /, then .vi ; vj / must be in T1 . Otherwise, .vi ; vj / would be in T2 and vi D p2 .vj / would create an inner node of T2 in vi . However, as B> .vi / D ¿, .vi ; p1 .vi // … E.G 0 /, contradicting the super-triangulation definition. Finally, if B> .vi / D ¿, and if vi is the clockwise last child of p0 .vi /, then .vi ; p1 .p0 .vi /// must be in T1 . Otherwise, .vi ; p1 .p0 .vi /// would be in T2 and vi D p2 .p1 .p0 .vi /// would create an inner node of T2 in vi . This contradicts the super-triangulation definition since, as B> .vi / D ¿, .vi ; p1 .vi // … E.G 0 /. Therefore, T1 is unique, and S is unique, completing the proof. t u 0

3 Encoding a Planar Graph with a Super-Triangulation We start this section with some useful properties for coding super-triangulations.

3.1 Properties of Super-Triangulations Straightforward from the equivalence between realizers and orderly trees of triangulations, we have the following basic property: Lemma 4. Let .T0 ; T1 ; T2 / be any realizer, and let v1 ; : : : ; vn be the clockwise preordering of the nodes of T 0 . For every vj with vi D p2 .vj / and vk D p1 .vj /, then i < j < k and neither vi nor vk is related to vj in T 0 . A cw-triangle is a triple of nodes .u; v; w/ of a realizer such that p2 .u/ D v, p1 .v/ D w, and p0 .w/ D u. In the realizer depicted in Fig. 2.3a, .u; v; w/ forms a cw-triangle, whereas the realizer of Fig. 2.3b has no cw-triangle. Let v1 ; : : : ; vn be the clockwise preordering of the nodes of a tree T . The subsequence vi ; : : : ; vj is a branch of T if it is a chain (i.e., vt is the parent of vt C1 for every i 6 t < j ), and if j i is maximal. Observe that vt C1 is necessarily the first child of vt because of the ordering of the vertices. Branches partition the nodes of T , and there is exactly one branch per leaf. The tree T 0 of a realizer .T0 ; T1 ; T2 / has the branch property if for all nodes vj and vi D p0 .vj /, either p2 .vj / D p2 .vi /, or vk D p2 .vj / with i < k < j (i.e., p2 .vj / is a descendant of vi clockwise before vj in T 0 ). An important feature of the branch property is that all the nodes of a given branch of T0 (maybe except the root

34

N. Bonichon et al.

of T0 ) must have the same parent in T2 . Indeed, vj and vi D p0 .vj / belong to the same branch implies that j D i C 1, and thus, because there is no index k such that i < k < j , p2 .vj / D p2 .vi / must hold. Definition 5 (minimal realizer). A minimal realizer is a realizer having the cwtriangle property. Lemma 5. Let S D .T0 ; T1 ; T2 / be any realizer. The following statements are equivalent: 1. 2. 3. 4.

S is a super-triangulation for some graph G. S has no cw-triangle. T i is well orderly in S , for every i 2 f0; 1; 2g. T i has the branch property in S , for every i 2 f0; 1; 2g.

Proof. Let 30 (resp. 40 ) denote the property 3 (resp. 4) for T 0 only. To prove 1 , 2 , 3 , 4, one can restrict our attention to 1 , 2 , 30 , 40 since “S has/has no cw-triangle” is a property stable by cyclic permutation of the trees. Thus, if 2 , 30 , then 2 , 3, and similarly for 40 . By definition of a super-triangulation, 1 ) 30 . Moreover, if S is a realizer with T 0 well orderly, then the realizer S is a super-triangulation of the underlying triangulation S (the four conditions trivially hold). Hence, 1 , 30 . To prove that 30 ) 2, assume that S has a cw-triangle .u; v; w/, and that T 0 is well orderly in S . As w D p1 .v/, then u is an ancestor of w in T 0 . It follows that v D p2 .u/ is a descendant of u in T 0 , contradicting Property 4. Hence, 30 ) 2. To prove that 40 ) 2, assume that S has a cw-triangle .u; v; w/, and that T 0 has the branch property. u D p0 .w/ and thus either p2 .u/ D p2 .w/ or p2 .w/ is a descendant of u contained inside the region bounded by the cw-triangle .u; v; w/. Clearly, p2 .u/ D p2 .w/ is impossible, and by the edge-ordering rule of realizers p2 .w/ is outside the region bounded by the cw-triangle .u; v; w/. Hence, 40 ) 2. Let v1 ; : : : ; vn be the clockwise preordering of the nodes of T 0 . To prove that 40 ) 3, assume that T 0 is orderly, but not well orderly in S . Thus, there is an edge .vp ; vj / with vj D p1 .vp / such that the nearest common ancestor between vp and vj in T 0 , say vt , is not vi D p0 .vj /. Consider the cycle C formed by the path in T 0 between vp and vj , and closed by the edge of .vp ; vj /. Let B be the bounded connected region of R2 nC . Let us calculate p2 .vi / and p2 .vj /. By the edge-ordering rule of realizers: (1) p2 .vj / … B; (2) p2 .vj / cannot belong to the path from vp to vt in T 0 with the edge .vj ; p2 .vj // lying outside B; and (3) p2 .vj / cannot belong to the path from vj to vt in T 0 since p2 .vj / is never a descendant of vj in T 0 (Property 4). Thus, p2 .vj / … B [ C . Again, from the edge-ordering rule of realizers, p2 .vi / must belong to B [ C . It follows that p2 .vi / D p2 .vj / or vk D p2 .vj / descendant of vi such that i < k < j is impossible, contradicting the branch property. Hence, 40 ) 3. It remains to show that 2 ) 40 . We assume that S has no cw-triangle. Let vj be any node, and let vi D p0 .vj / and let vl D p2 .vi /. We assume that k < l (T 0 has not the branch property), and we will show a contradiction.


35

Let vh D p0 .vl /, and let P be the path in T 0 between vh and vk . Note that vl … P because vl is not an ancestor of vk (k < l). Consider C the cycle obtained by traveling vi , vl , vh , P , vk , vj , vi , and let B be the bounded connected region of R2 n C . Let Q1 be the path in T 1 from vl to r1 , the root of T 1 . By the edge-ordering rule of realizers, the first edge of Q1 must belong to B [ C . Because r1 … B, Q1 must intersect C . By Property 4, the intersection must be in some vt with t > l. This intersection cannot be in vi from the edge-ordering rule of realizers. Since the nodes of P are vt ’s with t 6 max fh; kg < l, we have also that P \ Q1 D ¿. It follows that Q1 intersect C in vj . Let C1 be the cycle composed of Q1 , vj , vi , vl , and let B1 be the bounded connected region of R2 n C1 . Let us show that B1 [ C1 contains a cw-triangle. Either C1 is a cw-triangle and we are done with a contradiction, or consider the path Q2 in T 0 from p1 .vl / to r0 (the root of T 0 ). Similarly, Q2 must intersect C1 in vi . The cycle C2 traveling vi , vl , p1 .vl /, Q2 defines a connected region B2 of R2 n C2 with a number of faces at least one lower than the number of faces of B1 . Proceeding as previously, we can construct either a cw-triangle or a smaller region with the same inductive property. As the number of faces is finite, it follows that S has a cw-triangle: a contradiction completing the proof of 2 ) 40 , and thus of Lemma 5. t u Lemma 6. A tree of a realizer is uniquely determined by given the two others. Moreover, given the embedding of the two trees, it takes a linear time to construct the third one. Proof. By cyclic permutation of the trees, we only have to prove that the pair fT0 ; T1 g uniquely determines T2 . Consider the plane graph H composed of T0 [ T1 . To construct T2 , one needs to triangulate all the faces of H , except the outerface. Let F be any face of H , and let B be its boundary. We assume that F is not the outerface, and that T2 \ F contains at least one edge, i.e., that B has at least four nodes. Once F has been triangulated, B contains only three types of nodes depending on whether the directed edges of T2 \ F are leaving, entering, or whether no edge of T2 \ F are incident with them. Indeed, a node of B having leaving and entering edges of T2 \ F would contradict the edge-ordering rule of realizers. Let us call a sink a node of B having at least one entering edge of T2 \ F . Observe that B contains exactly one sink: zero sink is clearly impossible (T2 \ F ¤ ¿), and two or more sinks would provide a node with leaving and entering edges of T2 \ F , contradicting the edge-ordering rule of realizers. Clearly, once the sink has been located on B, there is only one way to triangulate F . It remains to show that there is only one place on B for a sink. Traveling B clockwise around F , the two adjacent edges of B at every node v form some cases named Xi Yj , where X; Y 2 fE; Lg and i; j 2 f0; 1g, with the following interpretation: v is in the Case E0 L1 if the edge clockwise before v in B is entering (E) in v and belongs to T0 , and if the edge clockwise after v in B is leaving (L) v and belongs to T1 . The other cases are defined similarly.

36

N. Bonichon et al.

Fig. 2.7 On the left, a tree of Bn (the root is indicated by a square). Then from left to right, the partial closure of the tree

Let us show that v is a sink if and only if v is in the Case L0 L1 . If v is a sink, then two adjacent edges of B with v must be leaving, and L0 L1 is the only possibility satisfying the edge-orientation rule of realizers. Now, assume that v occurs in the Case L0 L1 . Then, T2 \ F has no edge leaving v. If v has no entering and leaving edges in T2 \ F , then a node u 2 B adjacent to v must be a sink. The node u is therefore in the Case E1 Xi (if u is clockwise after v in B), or in the Case Yj E0 . However, we have already seen that a sink must be in the Case L0 L1 : a contradiction. Thus, v is a sink. It follows that the place of the unique sink on B is entirely determined by the edge pattern of B induced by the pair fT0 ; T1 g. Clearly, triangulate all the faces (by determining the sinks) takes linear time, completing the proof. t u In this section, we briefly recall a result from [33] about minimal realizers and plane trees. An encoding of well-orderly maps follows.

3.2 Minimal Realizers and Plane Trees The key point of this section is the definition of the operation of closure of planted tree to get a realizer (Fig. 2.7). A tree is planted if it is rooted on a leaf, that is a leaf is distinguished. Let Bn be the set of planted plane trees with n inner nodes and 2n leaves such that each node is adjacent to 2 leaves. Given a tree T in Bn , we can easily say that: • Its canonical orientation shall be toward the root for all inner edges and toward the leaf for all dangling edges. • Its canonical coloring described by the rule of Fig. 2.2a. A triple .e1 ; e2 ; e3 / of edges of a map M is an admissible triple if and only if: i/ e1 D .v0 ; v1 /, e2 D .v1 ; v2 /, and e3 D .v2 ; v3 / appear consecutively in the clockwise direction around the outer face and ii/ if v3 is a vertex of degree 1. Definition 6 (local closure). The local closure of M with respect to an admissible triple .e1 ; e2 ; e3 / is obtained by merging the leaf v3 on node v0 so as to create triangular face.


a l2

l2

37

b v0

v0

v2

l1

l1

v0

v1

v0

Fig. 2.8 The structure after a partial closure and the complete closure

The partial closure of a tree T of Bn is the map obtained by performing iteratively the local closure of any available admissible triple in a greedy way. As shown in [33], the local closure is well defined independently of the order of local closures. Moreover, all the bounded faces of the resulting map are triangular, and the outer face has the structure shown in Fig. 2.8a. In particular, in the partial closure of T , there are exactly two canonical nodes separating the dangling edges in the outer face into two parts. Each of these parts contains dangling edges of same color. A tree T is balanced if its root is one of the two canonical leaves. Finally, the complete closure of a balanced tree T is the map obtained from the partial closure of T by merging each remaining noncanonical leaf of each part into a root edge, as illustrated in Fig. 2.8b. Theorem 2 ([33]). Complete closure is a one-to-one correspondence between Bn2 and triangulations with n nodes. Observe that by construction the orientation of the dangling edge prevents the formation of cw-triangles implying that using Complete Closure, we get a minimal realizer. Corollary 1. Complete closure is a one-to-one correspondence between balanced trees with n 2 and minimal realizers of triangulation with n nodes. The following new lemma will serve to predict the entering edges created by complete closure at a node. Lemma 7. Let v be an inner node of a balanced tree B. Let e1 D .v; u/ and e2 D .v; w/ be two consecutive edges around v in clockwise order. During the closure algorithm, no edges will be inserted between e1 and e2 if and only if: (a) w is a leaf of B or (b) w is an inner node of B and the node t such that the edge e3 D .w; t/ is the next edge around w after e2 in clockwise order is a leaf of B.

38

N. Bonichon et al.

Proof. Let v an inner node of a balanced tree B. Let us consider two consecutive edges .v; u/, .v; w/ around v in clockwise order. If w is a leaf, then during the closure it will merge with a node w0 and close a triangular face enclosing the corner between .v; u/ and .v; w/. No other edge can thus arrive at this corner. Assume now that w is an inner node of B. Let .w; t/ be the next edge around w in clockwise order. If t is a leaf of B, then it will merge with u to form a triangular face, and again no edge can arrive in the corner between .v; u/ and .v; w/. In the other cases, .v; w/ is an inner edge followed by another inner edge .w; t/. Since an edge that forming a triangular face that encloses the corner between .v; u/ and .v; w/ must be from w, the corner is not enclosed. But at the end of the partial closure, there are no more pairs of consecutive inner edges: some edge must have arrived in the corner. t u Lemma 8. Let .T0 ; T1 ; T2 / be the minimal realizer encoded by a balanced tree B. A node v of B is a leaf of T2 if and only if v has no incoming edge colored 2 in B and, 1. The parent edge of v in B is colored 2 or 2. The parent edge of v in B is colored 1 or 3. The parent edge of v in B is colored 0 and v is the last child with an edge colored 0 in clockwise order around PB .v/ and a) The parent edge of PB .v/ is colored 0 or b) The parent edge of PB .v/ is colored 2. Proof. For the node v to be a leaf in T2 , it must have no incoming edge of color 2 in B, and no edge must be inserted between its outgoing edges of color 0 and 1. When the parent edge of v has color 2 or 1, the outgoing edge of color 0 connects to a leaf and Case (a) of the previous lemma ensures that no edge arrives between this outgoing edge of color 0 and the outgoing edge of color 1. When the parent edge of v has color 0, if the next edge in clockwise order around the parent PB .v/ of v in B is an outgoing edge (of color 1), then Case (b) of the previous lemma ensures that no edge of color 2 arrives. Finally, we need to check in the remaining cases that an incoming edge of color 2 indeed arrives between the two outgoing edges of color 0 and 1. This could happen if the corner we consider was part of the unbounded face after the partial closure. But in the remaining cases, both the edge .v; PB .v// and the next edge in clockwise order around PB .v/ are incoming. Since the form of the boundary after partial closure prohibits two consecutive incoming edges, the proof of the lemma is complete. u t

3.3 Representation of Planar Graphs with Binary Strings Along this section, we consider S D .T0 ; T1 ; T2 / be any super-triangulation of G, a connected planar graph with n nodes and m edges. We show how to use S to


39

efficiently represent G. Let `.B/ be the number of vertices of B corresponding to the leaves of T2 , where S D .T0 ; T1 ; T2 / is obtained by the complete closure of B. Theorem 3. Any well-orderly map with n nodes can be coded by a pair .B; W / where B is a balanced tree of Bn2 and W a bit string of length n C `.B/. Encoding and decoding takes linear time. The following lemmas describe in detail the key points of Theorem 3. Lemma 9. Let B be a balanced tree such that the corresponding supertriangulation S D .T0 ; T1 ; T2 / has i2 inner nodes in the tree T2 . The balanced tree B can be encoded with five binary strings S1 ; S2 ; S3 ; S4 , and S5 and four integers a0 ; a00 ; a1 ; i2 6 n such that: 0 1 1 0 , #S2 D na , #S3 D nCa , #S4 D a1 Caa00 Ca0 and #S5 D #S1 D ina a1 a00 2 a0 na1 a0 0 . na a0 i 1

0

2

Proof. Let B be a colored balanced tree. We partition the nodes of B in the following way: • A1 : the set of nodes v such that the edge .v; PB .v// is colored 1. • A2 : the set of nodes v such that the edge .v; PB .v// is colored 2. • A00 : the set of nodes v and such that the edge .PB .v/; PB .PB .v// is colored either 0 or 2, and such that v is the last child in clockwise order with the edge .v; PB .v// is colored 0. • A0 : the set of nodes that are not in the previous sets. Note that the root of B is in A0 and for every node v of A0 , the edge .v; PB .v// is colored 0. Assume that we are coding the balanced tree B. The only information we need, for each node in the prefix clockwise order, is its number of children in A0 , A00 , A1 , and A2 . In order to encode efficiently a well-orderly map, we need to introduce another parameter in our encoding. Let I2 be the set of nodes of B that will be inner nodes in the tree T2 of the corresponding realizer S D .T0 ; T1 ; T2 /. We give some preliminary remarks: Nodes of A1 cannot have children in A00 . Every node of A0 [ A00 [ A2 has at most one child in A00 . A0 I2 (see Lemma 8). Every node of A00 [ A1 [ A2 which is also in I2 has at least one child in A2 (see Lemma 8). 5. Every node of V n A1 can have children in A0 only if it has a child in A00 . 6. Only nodes of I2 can have children in T2 . 1. 2. 3. 4.

To encode the balanced tree, we will build five binary strings. With these strings we will determine, for each node, its number of children in each subset. The first string, S1 , tells which node belongs to I2 . Since all the nodes of A0 are in I2 (see Remark 3), S1 stores the information for all the other nodes. Hence for each node of V n A0 , the corresponding bit is set to 1 if the node belongs to I2 and is set to 0 otherwise. Hence, the string S1 contains n a0 bits and i2 a0 1’s.

40

N. Bonichon et al.

The second string S2 is used to determine whether a node has a child in A00 . Since all the nodes of A1 have a child in A00 (see Remark 1), S2 stores this information for all the other nodes: the corresponding bit is set to 1 if the node has one child in A00 and to 0 otherwise. Hence, the string S2 contains n a1 bits and a00 1’s. The string S3 stores, for each node, its number of children in A1 in a “Lukasiewicz” way. For each v node of B in the prefix clockwise order, we append to S3 as many 1’s as the number of children of v in A1 and then we insert a 0. Hence, the string S3 contains n C a1 bits and a1 1’s. The string S4 stores the number of children in A0 . This information has to be stored for each node of A1 and for each node that has a child in A00 (see Remark 5). Hence for each of these nodes, we proceed as for the string S3 . Hence, the string S4 contains a1 C a00 C a0 bits and a0 1’s. The string S5 helps to determine the number of children in A2 . We only need to store this information for the nodes of I2 (see Remark 6). Moreover, for these nodes that are in A0 [ A00 [ A2 , we already know that they have at least one child in A2 ; so we only need to count the other 1’s. Hence for each of these nodes, we proceed as for the strings S3 and S4 . We obtain a string i2 C .a2 .i2 a0 // D n a1 a00 bits with a2 .i2 a0 / D n a1 a00 i2 1’s. t u Lemma 10. Let H be a well-orderly map with n nodes and m edges. H can be encoded with six binary strings (five for the minimal realizer and a last one to store the missing edges) and four integers a0 ; a1 ; a00 ; i2 2 Œ0; n such that: 0 1 1 na1 a0 0 0 , #S2 D na , #S3 D nCa , #S4 D a1 Caa00 Ca0 , #S5 D na a , #S1 D ina 0 a00 a1 2 a0 1 0 i2 2ni 2 #S6 D mni2 . Proof. With S1 S5 a minimal realizer is encoded (Lemma 9). The last string indicates the edges to delete to rebuild the well-orderly map: for each v, one bit is used to indicate if the edge .v; p2 .v// has to be removed, and for each leaf v of T2 , one bit is used to indicate if the edge .v; p1 .v// has to be removed. t u We present here a variant of the Pagh’s compressor [32]. We denote by #S the number of binary strings having the same length and the same number of ones than S . Lemma 11. Every binary string S of length n can be coded into a binary string of length log2 .#S /CO.n log log n= log n/. Moreover, knowing n, coding and decoding S can be done in linear time, assuming a RAM model of computation on words of ! > log2 n bits. Proof. The main idea is to split S into blocks of equal size b, and to code each block optimally. Each block encoding takes a time exponential in b. However, the code of all possible blocks can be tabulated once in time O.2b / D O.n/, for suitable b small enough. Optimality of the coding derives from optimality of each block by super-additivity of binomials. More precisely, we proceed as follows.


41

Let b D blog2 n log2 log2 nc. Note that 2b 6 n= log2 n. Standard arithmetic operations on integers in the range Œ0; 2b / can be done in constant time since ! > b, by loading each integer into a word of length ! and padded with left extra zeros. Let kp denote pb . We need the construction of some tables. ˙ We first construct a table L such that for every p 2 Œ0; b, LŒp D log2 kp . All the numbers k0 ; : : : ; kb are thanks to the Pascal’s method, namely using computed, i 1 C for all i 6 b and j 6 min fi; pg. This uses iteratively the formula ji D ji 1 1 j P O.b 2 / b-bit numbers. In total, the construction of L is done in O.b 2 C p log kp / D 2 O.log ˙ n/ time, as it costs no more than O.log kp / D O.b/ time to compute log kp , that is the position of the leading bit of kp in its binary representation. We construct a table P of integers in the range Œ0; b such that for every i 2 Œ0; 2b /, P Œi is the number of ones in the binary representation of i . The table P can b be constructed in time can be reduced p and space O.b2 / D O.n/. However, the time b=2 to O.b2 / D O. n log n / (and even smaller) using a table P 0 for half-words of db=2e bits. Indeed, we have P Œi D P 0 Œi=2bb=2c C P 0 Œi mod 2bb=2c . For each p 2 f0; : : : ; bg, we compute a table Dp (used for decoding) such that, for every i 2 Œ0; kp /, Dp Œi is a distinct binary string of length b having p ones. Strings of Dp are lexicographically ordered. Generating all Dp ’s costs O.2b / D O.n= log n/ time and O.b2b / D O.n/ space by running all binary strings s 2 Œ0; 2b / by increasing value, and filling the right entry DP Œs Œip (and updating the current index ip ). Finally, we construct a table C (used for coding) such that for every s 2 Œ0; 2b /, C Œs denotes the index i such that Dp Œi D s, where ˙ p D P Œs. The index i D C Œs is stored on b bits, although only the LŒp D log2 kp least significant bits of i are useful since i 2 Œ0; kp /. To construct C , we iterate for all p 2 Œ0; b and all i 2 Œ0; kp /: C ŒDp Œi D i . Once Dp and P have been computed, constructing C P costs O. bpD0 kp / D O.2b / D O.n= log n/ of time and O.b2b / D O.n/ of space. Let q D bn=bc be the number of blocks of b bits in S . If b does not divide n, the last n mod b bits will be treated separately. For the coding and decoding procedure, we iterate on each block of S times (so q times) the following steps: 1. Read from S the next b-bit block s, manipulated as an index of Œ0; 2b / 2. Write in S 0 (the coding string) the value P Œs as a binary number on dlog2 be bits 3. Write in S 0 the string composed of the LŒs most significant bits of C Œs We end the coding process by writing in S 0 the n mod b remaining bits of S (if any). The decoding procedure of S 0 in S is: 1. Read from S 0 the dlog2 be bits to form the value p 2. Read from S 0 the next LŒp bits, representing an integer i 2 Œ0; kp / 3. Write in S the string Dp Œi We end the decoding process by writing in S the n mod b remaining bits of S 0 (if any).

42

N. Bonichon et al.

Coding and decoding procedures clearly take O.q/ D O.n= log n/ time, once the tables L; P; Dp ; C have been generated. The correctness of the coding and decoding is clear from symmetry of the above procedures. It remains to show that the length of S 0 does not exceed log2 .#S / C O.n log log n= log n/. Let pi be the number of ones in the i th b-bit block of S , for i 2 f1; : : : ; qg. From the˙coding procedure, the number of bits written in S 0 for the i th block is: dlog2 be C log2 kpi . Summing over all the blocks, we obtain the following upper bound for the length of S 0 : q X ˙ dlog2 be C log2 kpi C .n mod b/ D

q X

i D1

i D1

Observe that by super-additivity q Y

kpi D

i D1

! log2 kpi

C O.b C q log b/:

a a0 aCa0 b 0 6 bCb0 , so b

q Y i D1

b pi

! 6

bq P i pi

! D #S;

where S is the string composed of the first bq bits of S . Since the length and the number of ones between S and S differ by at most b, it follows that j log2 .#S / log2 .#S/j D O.b log n/ D O.log2 n/. Therefore, we have that the length of S 0 is no more than: ! q Y log2 kpi C O.b C q log b/ 6 log2 .#S / C O.n log log n= log n/ i D1

as claimed, completing the proof.

t u

4 Entropy Analysis The length of the coding of well-orderly map depends of the number of the edges of the well-orderly map. The following two results are obtained from the analysis of the length of the code given in Lemma 10. The length of this code depends on the number of edges of the well-orderly map (see Fig. 2.9). Theorem 4. Every connected planar graph with n nodes and m edges can be encoded in linear time with at most 4:91n C o.n/ bits or 2:82m C o.m/ bits. In particular, the number of well-orderly maps with n nodes (resp. with m edges) is at most 24:91nCo.n/ (resp. 22:82mCo.m/ ). Proof (Sketch). From Lemma 10, we obtain an explicit coding composed of six binary strings S1 ; S2 ; :::; S6 and four integers a0 ; a1 ; a00 ; i2 .


a

43

b

5.5

3 2.8

5

2.6 4.5

2.4

4

2.2 2

3.5

1.8 1.6

3

1.4

2.5 2

1.2 1

1.2 1.4 1.6 1.8

2

2.2 2.4 2.6 2.8

3

1 1

1.2 1.4 1.6 1.8

2

2.2 2.4 2.6 2.8

3

Fig. 2.9 (a) Number of bits necessary to encode a well-orderly map with m D ˛n edges, where 1 6 ˛ 6 3. (b) Coding analyses: Number of bits per edges of a well-orderly map with m D ˛n edges, where 1 6 ˛ 6 3

Thanks to Lemma 11, we can encode in linear time a planar graph with at most log2 .#S1 /Clog2 .#S2 /Clog2 .#S3 /Clog2 .#S4 /Clog2 .#S5 /Clog2 .#S6 /CO.log n/ bits. Computing the maximum length of the codes (over all parameters a0 ; a1 ; :::; i2 and m or n) : ( W D W .n; m/ D max

a0 ;a1 ;:::;i2

! ! ! n a0 n a1 n C a1 log2 C log2 C log2 i2 a0 a00 a1 ! ! a1 C a0 C a00 n a1 a00 C log2 C log2 a0 n a1 a00 i2 !) 2n i 2 C log2 C O.log n/; m n i2

we obtain (see Fig. 2.9a) that W 6 4:91n C o.n/. Since G is connected, we have n 1 6 m 6 3n 6 and so log n D log m C O.1/. Hence, we also have (see Fig. 2.9b) that W 6 2:28m C o.m/. t u Theorem 5. Almost all unlabeled connected planar graphs on n nodes have at least 1:85n edges and at most 2:44n edges. Proof. (sketch). Our code can be parameterized with the number of edges. The length of the coding is no more than W .m; n/ bits. Using a reduction from arbitrary planar graphs to connected planar graphs, we can apply our upper bound. Combined with the 4:767n bit lower bound derived from the number of labelled planar graphs of [18], we derive two numbers ˛1 D 1:85 and ˛2 D 2:44 such that for a connected planar graph with ˛n edges, ˛1 6 ˛ 6 ˛2 , our representation is below 4:767n (See Fig. 2.9a). t u

44

N. Bonichon et al.

Fig. 2.10 Representation of a non-connected planar graph Q v/ G by a triple .k; G;

G

˜ G

v2 v1

v3

v4 v5

v

v6

The bound proposed here was for connected planar graphs. This bound can also be apply on planar graphs (not necessarily connected). Remind that p.n/ is the number of unlabeled n-node planar graphs. Let q.n/ denote the number of unlabeled connected planar graphs on n nodes. To relate p.n/ and q.n/, we represent every planar graph G with k > 1 connected components by Q v/, where GQ and v are defined as follows. Let vi be any non-cut-vertex a triple .k; G; of the i th component of G. (Recall that a cut-vertex of a graph is a node whose removal strictly increases the number of connected components. A leaf of a tree being not a cut-vertex, it is clear that every connected graph has a non-cut-vertex). We merge all the connected components of G by identifying all the vi ’s into a single node v as shown in Fig. 2.10. Q v/ by splitting Clearly GQ is planar and connected. One can obtain G from .k; G; 0 Q v in G. All the k 6 k connected components obtained by this way are included in G (there is no risk to disconnect a single connected component of G as vi ’s are not cut-vertices of each connected component of G). To fully recover G, we may add k k 0 isolated nodes. The number of nodes of GQ is n.k 0 k/k 0 C1 D nk C1. From this representation, it follows that: p.n/ 6

n X

k q.n k C 1/ .n k C 1/ 6 n3 q.n/

(2.1)

kD1

as q.n/ clearly increases with n. In [7], it is shown that p the number of n-node well-orderly maps is upper bounded p p nCo.n/ with D 8=. 189 C 114 3 6 3 9/ 30:0612. From the above by discussion, we have: Theorem 6. The number p.n/ of unlabeled n-node planar graphs, for n large enough, satisfies p.n/ 6 nCo.n/ 6 24:91n :

5 Summary and Conclusion Counting the number of unlabeled planar n-node graph is still an open problem. In this chapter, we show the best known upper bound that equals 30:0612nCo.n/. Moreover, we propose a compact encoding of planar graphs using a new combinatorial


45

object called well-orderly map with log2 30:0612 4:91 bits per node. The coding and the decoding can be done in linear time. Since a planar graph can be represented by several well-orderly maps, a possible way to get a smaller bound would be to define a smaller class of planar maps. Finding a way to assign to any planar graph at most 2o.n/ planar embeddings is enough to get the asymptotic value of constant growth. Another direction consists in counting the number of labeled planar graphs and computing the average number of symmetries (automorphisms) per graph. The first step has been done by Gimenez and Noy [18], but the knowledge of the average number of symmetries is still a problem.

References 1. Alonso, L., Rémy, J.L., Schott, R.: A linear-time algorithm for the generation of trees. Algorithmica 17(2), 162–182 (1997) 2. Banderier, C., Flajolet, P., Schaeffer, G., Soria, M.: Planar maps and airy phenomena. In: 27th International Colloquium on Automata, Languages and Programming (ICALP), vol. 1853 of Lecture Notes in Computer Science, pp. 388–402. Springer, New York (2000) 3. Barcucci, E., del Lungo, A., Pergola, E.: Random generation of trees and other combinatorial objects. Theor. Comput. Sci. 218(2), 219–232 (1999) 4. Bodirsky, M., Kang, M.: Generating random outerplanar graphs. In: 1st Workshop on Algorithms for Listing, Counting, and Enumeration (ALICE) (2003) 5. Bonichon, N.: A bijection between realizers of maximal plane graphs and pairs of non-crossing Dyck paths. In: Formal Power Series & Algebraic Combinatorics (FPSAC) (2002) 6. Bonichon, N., Gavoille, C., Hanusse, N.: An information-theoretic upper bound of planar graphs using triangulation. In: 20th Annual Symposium on Theoretical Aspects of Computer Science (STACS), vol. 2607 of Lecture Notes in Computer Science, pp. 499–510. Springer, New York (2003) 7. Bonichon, N., Gavoille, C., Hanusse, N., Poulalhon, D., Schaeffer, G.: Planar graphs, via wellorderly maps and trees. Graph. Combinator. 22, 1–18 (2006) 8. Chiang, Y.T., Lin, C.C., Lu, H.I.: Orderly spanning trees with applications to graph encoding and graph drawing. In: 12th Symposium on Discrete Algorithms (SODA), pp. 506–515. ACM-SIAM (2001) 9. Chiba, N., Nishizeki, T., Abe, S., Ozawa, T.: A linear algorithm for embedding planar graphs using pq-trees. J. Comput. Syst. Sci. 30(1), 54–76 (1985) 10. Chiba, N., Nishizeki, T., Abe, S., Ozawa, T.: A linear algorithm for embedding planar graphs using pq-trees. J. Comput. Syst. Sci. 30(1), 54–76 (1985) 11. Chuang, R.C.N., Garg, A., He, X., Kao, M.Y., Lu, H.I.: Compact encodings of planar graphs via canonical orderings and multiple parentheses. In: Guldstrand Larsen, K., Skyum, S., Winskel, G. (eds.) 25th International Colloquium on Automata, Languages and Programming (ICALP), vol. 1443 of Lecture Notes in Computer Science, pp. 118–129. Springer, New York (1998) 12. Denise, A., Vasconcellos, M., Welsh, D.J.A.: The random planar graph. Congressus Numerantium 113, 61–79 (1996) 13. Diestel, R.: Graph Theory, 2nd edn., vol. 173 of Graduate Texts in Mathematics. Springer, New York (2000) 14. Epstein, P., Sack, J.R.: Generating triangulations at random. ACM Trans. Model. Comput. Simul. 4, 267–278 (1994) 15. Frederickson, G.N., Janardan, R.: Efficient message routing in planar networks. SIAM J. Comput. 18(4), 843–857 (1989)

46

N. Bonichon et al.

16. Gavoille, C., Hanusse, N.: Compact routing tables for graphs of bounded genus. In: Wiedermann, J., van Emde Boas, P., Nielsen, M. (eds.) 26th International Colloquium on Automata, Languages and Programming (ICALP), vol. 1644 of Lecture Notes in Computer Science, pp. 351–360. Springer, New York (1999) 17. Gerke, S., McDiarmid, C.J.H.: On the number of edges in random planar graphs. Combinator. Probab. Comput. 13, 165–183 (2004) 18. Giménez, O., Noy, M.: Asymptotic enumeration and limit laws of planar graphs. J. Am. Math. Soc. 22, 309–329 (2009) 19. He, X., Kao, M.Y., Lu, H.I.: A fast general methodology for information-theoretically optimal encodings of graphs. SIAM J. Comput. 30(3), 838–846 (2000) 20. Keeler, K., Westbrook, J.: Short encodings of planar graphs and maps. Discrete Appl. Math. 58, 239–252 (1995) 21. Khodakovsky, A., Alliez, P., Desbrun, M., Schröder, P.: Near-optimal connectivity encoding of 2-manifold polygon meshes. Graph. Model. (2002). To appear in a special issue 22. King, D., Rossignac, J.: Guaranteed 3.67V bit encoding of planar triangle graphs. In: 11th Canadian Conference on Computational Geometry (CCCG), pp. 146–149 (1999) 23. Lipton, R.J., Tarjan, R.E.: A separator theorem for planar graphs. SIAM J. Appl. Math. 36(2), 177–189 (1979) 24. Liskovets, V.A., Walsh, T.R.: Ten steps to counting planar graphs. Congressus Numerantium 60, 269–277 (1987) 25. Liu, Y.: Enumeration of simple planar maps. Utilitas Math. 34, 97–104 (1988) 26. Lu, H.I.: Improved compact routing tables for planar networks via orderly spanning trees. In: 8th Annual International Computing & Combinatorics Conference (COCOON), vol. 2387 of Lecture Notes in Computer Science, pp. 57–66. Springer, New York (2002) 27. Lu, H.I.: Linear-time compression of bounded-genus graphs into information-theoretically optimal number of bits. In: 13th Symposium on Discrete Algorithms (SODA), pp. 223–224. ACM-SIAM (2002) 28. McDiarmid, C.J.H., Steger, A., Welsh, D.J.A.: Random planar graphs. J. Combin. Theor. B 93, 187–205 (2005) 29. Munro, J.I., Raman, V.: Succinct representation of balanced parentheses, static trees and planar graphs. In: 38th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 118–126. IEEE Computer Society Press (1997) 30. Nishizeki, T., Chiba, N.: Planar Graphs: Theory and Algorithms. North-Holland Mathematics Studies 140, Amsterdam (1988) 31. Osthus, D., Prömel, H.J., Taraz, A.: On random planar graphs, the number of planar graphs and their triangulations. J. Combin. Theor. B 88, 119–134 (2003) 32. Pagh, R.: Low redundancy in static dictionaries with constant query time. SIAM J. Comput. 31(2), 353–363 (2001) 33. Poulalhon, D., Schaeffer, G.: Optimal coding and sampling of triangulations. In: 30th International Colloquium on Automata, Languages and Programming (ICALP), vol. 2719 of Lecture Notes in Computer Science, pp. 1080–1094. Springer, New York (2003) 34. Rossignac, J.: Edgebreaker: Connectivity compression for triangle meshes. IEEE Trans. Visual. Comput. Graph. 5(1), 47–61 (1999) 35. Schaeffer, G.: Random sampling of large planar maps and convex polyhedra. In: 31st Annual ACM Symposium on Theory of Computing (STOC), pp. 760–769 (1999) 36. Schnyder, W.: Embedding planar graphs on the grid. In: 1st Symposium on Discrete Algorithms (SODA), pp. 138–148. ACM-SIAM (1990) 37. Thorup, M.: Compact oracles for reachability and approximate distances in planar digraphs. In: 42nd Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 242–251. IEEE Computer Society Press (2001) 38. Turán, G.: Succinct representations of graphs. Discrete Appl. Math. 8, 289–294 (1984) 39. Tutte, W.T.: A census of planar triangulations. Cana. J. Math. 14, 21–38 (1962) 40. Yannakakis, M.: Embedding planar graphs in four pages. J. Comput. Syst. Sci. 38, 36–67 (1989)

Chapter 3

Probabilistic Inference Using Function Factorization and Divergence Minimization Terence H. Chan and Raymond W. Yeung

Abstract This chapter addresses modeling issues in statistical inference problems. We will focus specifically on factorization model which is a generalization of Markov random fields and Bayesian networks. For any positive function (say an estimated probability distribution), we present a mechanical approach which approximates the function with one in a factorization model that is as simple as possible, subject to an upper bound on approximation error. We also rewrite a probabilistic inference problem into a divergence minimization (DM) problem where iterative algorithms are proposed to solve the DM problem. We prove that the well-known EM algorithm is a special case of our proposed iterative algorithm. Keywords Divergence distance • Factorization • Hammersley–Clifford theorem • Markov random field • Maximum likelihood estimation MSC2000 Primary 62J12; Secondary 94A15, 91B70.

1 Introduction The typical goal of a statistical inference problem is to estimate a function (e.g., the probability distribution of a network of random variables) from a set of observed data. In many cases, the function being estimated is assumed belonging to some statistical models. There are several reasons for imposing a modeling assumption. First, it simplifies the inference problem. For example, assuming the underlying probability distribution to be Gaussian, the statistical inference problem

T.H. Chan () Institute for Telecommunications Research, University of South Australia, Adelaide, SA 5095, Australia e-mail: [email protected]; [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 3, © Springer Science+Business Media, LLC 2011

47

48

T.H. Chan and R.W. Yeung

is reduced to estimation of the mean and variance of the underlying distribution. Furthermore, simpler models are also preferable in practice. When dealing with a complicated function involving multiple variables, it is always desirable to factorize such a function into a product of “local” functions to reduce memory space for storing the function and to make updates of estimation more efficiently. Second, the way we choosing the model reflects our expectation or prior knowledge about the function itself. For instance, Bayesian networks can model the causality relations among a network of dependent events. The choice of a particular model reflects our prior knowledge on how events triggering each other. It is a fundamental question in statistical inference about how to choose a model that is good enough to explain the observable outcomes and is simple enough to be used in practice. This chapter aims to answer this question in the context of “factorization models”. It consists of two main parts. In the first part, we consider a very simple statistical inference problem in which the observed data are complete such that the empirical distribution P .x1 ; : : : ; xn / is a sufficient statistics and can be determined from the observed data. The inference problem thus becomes finding a distribution in a model that best explains P . We will consider a special class of models called factorization models, which generalize Markov random fields (also known as Gibbs fields [15, 17]) and Bayesian networks[16]. The main questions to be answered here are: How a global function P can be factorized into a product of local functions and what the simplest factorization model it belongs to? Furthermore, even if P is already factorized in its simplest form, the model can still be too complicated to work with in practice. In this case, we aim to approximate P with one that can be factorized as simple as possible, while ensuring that the resulting approximation error is acceptable. In the second part, we consider a more general setup in which the observed data are incomplete, and hence it is not possible to compute the empirical distribution from the data. We will model such statistical inference problem as a divergence minimization problem and will propose an iterative algorithm to solve it. The organization of this chapter is as follows. In Sect. 2, we will begin with an introduction of Markov random fields and Bayesian networks. Then we will propose factorization models as a generalization. In Sect. 3, we propose various transforms, using which we can determine how a global function can be factorized. Theorem 2 in this section subsumes the Hammersley–Clifford Theorem [1, 19], a fundamental result in graphical models. We also address the issue of how to approximate a global function with one in a simpler factorization model. Bounds on approximation errors are given, via which we can find the simplest approximation without introducing excessive approximation errors. Section 4 considers a statistical inference problem subject to incomplete data. We will formulate the problem as a divergence minimization (DM) problem and will propose algorithms to solve for the minimum. Finally, in Sect. 5, we will examine the maximum likelihood estimation (MLE) problem. We will show that MLE is closely related to divergence minimization, and that our algorithm proposed in Sect. 4 indeed generalizes the well-known EM algorithm [13, 20, 23].

3 Probabilistic Inference Using Function Factorization and Divergence Minimization

49

Notations Let N D f1; 2; : : : ; ng and P.N / be the power set of N . Consider a set of random variables Xi indexed by N , where Xi takes values from a nonempty finite Q set Xi of cardinality jXi j. For any a 2 P.N /, Xa denotes the Cartesian product i 2a Xi . When a D N , Xa is simply denoted by X . Let x D .x1 ; : : : ; xn / 2 X . For any a 2 P.N /, xa D .xi ; i 2 a/, which is an element in Xa . X Let RX , RX 0 , and RC be, respectively, the set of real, nonnegative, and positive functions defined on X . For any a N , a is its set complement defined as N na. For any p 2 RX and subset a of N , p.xa / is defined as the average of p.xa ; ya / over ya 2 Xa . In other words: X p.xa / D p.xa ; ya /=jXa j: (3.1) ya 2Xa

It is worth to note that if p.x/ depends only on xa , then p.x/ D p.xa / for all x 2 X .

2 Models A typical question in statistical inference is to estimate the underlying probability distribution of a set of random variables from a set of observable data. Usually, the underlying distribution is assumed to be from a specific model. In this section, we will introduce some of the most commonly used models. The first one is Markov random field or Markov network, dating back to 1925 when German physicist Ernst Ising used it to model spins in ferromagnetic materials [17]. Definition 1 (Markov random field). Let G D .V; E/ be an undirected graph with a set of vertices V D f1; 2; : : : ; ng. Then G is called a Markov random field for X1 ; : : : ; Xn if Xv1 and Xv2 are conditionally independent given Xı for any subset ı V and v1 ; v2 62 ı such that the nodes v1 and v2 are separated in the subgraph of G obtained by removing all vertices in ı and any of their connecting edges. Example 1. Let G be the network depicted in Fig. 3.1a. It is a Markov random field for fX1 ; X2 ; X3 ; X4 g if (1) X1 and X3 are conditionally independent, given X2 ; X4 and (2) X2 and X4 are conditionally independent given X1 ; X3 . According to the Definition 1, G is a Markov random field for a set of random variables if how nodes in G are separated from each other is consistent with the conditional independence structure within the set of variables.

a1

4

b1

2

3

2

Fig. 3.1 A Markov random field and a Bayesian network

3

4

50


Theorem 1 (Hammersley–Clifford Theorem [19]). Consider any positive probability distribution P .x1 ; : : : ; xn /. A graph G is a Markov random field for X1 ; : : : ; Xn if and only if there exists functions QsQ .xi W i 2 s/ where S is a collection of cliques of G such that P .x1 ; : : : ; xn / D s2S Qs .xi W i 2 s/. While Hammersley–Clifford Theorem does not hold if P is not positive, the theorem motivates a slightly different definition for Markov random field based on how the underlying probability distribution factorizes: A Markov random field is an undirected graph G together with a set of functions Qs .xi W i 2 S /; s 2 S , where (1) S is the set of cliques in G, and (2) the joint distribution of the variables is given by: 1 Y Pr.x1 ; : : : ; xn / D Qs .xi W i 2 s/; Z s2S where Z is a normalizing constant. The resulting joint distribution is usually called the Gibbs measure, while Z is called the partition function. Bayesian networks [16, 22] are another class of common graphical models defined on directed acyclic graphs. They are an important tool for many statistical inference problems, including pattern classification and unsupervised learning [16, 18]. Definition 2 (Bayesian network). Let G D .V; E/ be a directed acyclic graph. Each node i in V is associated with a random variable Xi . Let pa.i / be the set of “parents” of node i . Then the probability distribution represented by the Bayesian network G assumes the form: Pr.x1 ; : : : ; xn / D

n Y

Pr.xi jxpa.i / /:

i D1

Example 2. Let G be the directed acyclic graph in Fig. 3.1b and is a Bayesian network for random variables fX1 ; X2 ; X3 ; X4 g. Then: Pr.x1 ; x2 ; x3 ; x4 / D Pr.x1 / Pr.x2 / Pr.x3 jx1 ; x2 / Pr.x4 jx3 /:

(3.2)

Both Markov random fields and Bayesian networks are defined by factorization of a probability distribution into a product of “local functions” with respect to a graph. In the following, we extend the idea by proposing a new model which is defined only based on how a function factorizes without referring to any underlying graphical structure. Definition 3 (Factorization model). Let Ã be the set of all nonempty subsets of P.N / and 2 Ã, i.e., is a nonempty collection of subsets of N . A factorizationQmodel M is the set of positive functions P 2 RX C that assume the form P .x/ D a2 Qa .xa /. In other words, P .x/ can be factorized into a product of local functions Qa .xa / for a 2 .


51

Example 3. Let D ff1; 2g; f2; 3g; f1; 3gg. Then a function p belongs to the factorization model M if p can be factorized as below for some local functions Q1;2 ; Q2;3 , and Q1;3 . p.x/ D Q1;2 .x1;2 /Q2;3 .x2;3 /Q1;3 .x1;3 /: Let 2 Ã. Define: max./ D fa 2 W 8b 2 ; a b implies a D bg; D fa N W 9b 2 s.t. a bg:

(3.3) (3.4)

In other words, max./ is the set of maximal elements in in the usual set-inclusion sense, and is the collection of subsets of N that are subsets of some elements in . It is trivial to prove the following. Lemma 1. Let ; 2 Ã. Then: 1. M D Mmax./ D M . 2. If , then M M .

3 Function Approximation Using Factorizations Consider a simple statistical inference problem where it is required to estimate the underlying probability distribution P .x1 ; : : : ; xn / of random variables fX1 ; : : : ; Xn g from a set of experimental data. Assume that one can determine an empirical distribution PO from the experimental data, and that the underlying distribution P .x1 ; : : : ; xn / is in a given factorization model M . In most cases, PO is not contained in the given model M . Therefore, the question is: how to find an estimate in M that is “nearest” PO (or roughly speaking, that best “explains” PO ). In some other scenarios, we may not even know which model the underlying distribution belongs to. The challenge thus is: how to determine the most simplest model that is good enough to explain the experimental outcomes. In this section, we aim to address these two problems using an approach called function factorization [2–4].

3.1 Factorization of Functions First, we will state without proof a simple variation of Mobiüs inversion, which is an important analytical tool for deriving the results in this section. ˙ a be a positive real number for all a 2 Proposition 1 (Mobius ¨ Inversion). Let P.N /. Then: Y ˚b ; 8a 2 P.N / a D ba

52


if and only if ˚a D

Y ba

.1/janbj

b

;

8a 2 P.N /:

As a corollary, by letting a D C > 0 for all a 2 P.N /, we have: ˚a D

Y

C

.1/janbj

D

ba

1 if a ¤ ; C if a D ;:

y Definition 4 (˚ y -transform). Fix any y 2 X and P 2 RX C . The ˚ -transform of y P is a set of component functions f˚a ŒP 2 RX W a 2 P.N /g such that: C

˚ay ŒP .x/ D

Y

P .xb ; yb /.1/

janbj

:

(3.5)

ba y

y

Clearly, ˚a ŒP .x/ depends only on xa and hence ˚a ŒP .x/ can simply be y y y y denoted by ˚a ŒP .xa /. Furthermore, ˚a ŒPQ D ˚a ŒP ˚a ŒQ, where the function PQ is defined by PQ.x/ D P .x/Q.x/ for all x 2 X . ˙ any y 2 X and P 2 RX Theorem 2 (Minimal factorization). For C , let , fa y N W ˚a ŒP ¤ 1g, where 1 denotes the all one vector. Then Q y (i) P .x/ D a2 ˚a ŒP .xa /, and (ii) M is the “minimal factorization model” containing P . In other words, if P 2 M , then (and consequently, ). y

Proof. QTo prove (i ), let a D P .xa ; ya / and ˚a D ˚a ŒP .xa /. By Proposition 1, a D ba ˚b and hence: P .x/ D N D

Y

˚ay ŒP .xa / D

aN

Y a2

˚ay ŒP .xa /:

To prove (ii), consider any function Q in RX C that depends only on xa and a subset b such that b 6 a. For any subset c 2 P.N /, it can be uniquely written as the union of two disjoint subsets d; e where d D c \ a and e D cna. Clearly, c b if and only if d b \ a and e bna. Hence: y

˚b ŒQ.x/ D D

Y cWcb

Y

Q.xc ; yc /.1/ Y

jbncj

Q.xd ; xe ; y.d [e/ /.1/

(3.6) jbn.d [e/j

(3.7)

eWebna d Wd b\a .a/

D

Y

Y

eWebna d Wd b\a

Q.xd ; ye ; y.d [e/ /.1/

jbn.d [e/j

(3.8)


D

Y

Y

Q.xd ; yd /.1/

j.b\a/nd j .1/j.bna/nej

53

(3.9)

eWebna d Wd b\a

D

Y eWebna

0

1.1/j.bna/nej

Y

@

Q.xd ; yd /.1/

j.b\a/nd j

A

(3.10)

d Wd b\a

.b/

D 1;

(3.11)

where (a) follows from that Qa .x/ does not depend on xe , and (b) follows from that bna is nonempty and Proposition 1. Now suppose that P 2 M . ThenQthere exists a set of positive functions fQa .xa /; a 2 g such that P .x/ D a2 Qa .xa /. Suppose b 62 . Then, bna y is nonempty for all a 2 . As Qa .x/ depends only on xa , ˚b ŒQa D 1 for all a 2 . Thus: " # Y y y ˚b ŒP D ˚b Qa (3.12) D

Y a2

a2 y

˚b ŒQa

D 1:

(3.13) (3.14)

Therefore, b 62 or equivalently, .

t u

Corollary 1. Let P be a positive probability distribution for random variables y fX1 ; : : : ; Xn g and D fa N W ˚a ŒP ¤ 1g. Suppose X˛ and Xˇ are independent given X.˛;ˇ/ . If b is a set such that both b \ ˛ and b \ ˇ are nonempty, then b 62 . Remark 1. Using Corollary 1, the Hammersley–Clifford Theorem can be proved by showing that b 62 if b is not a subset of a clique in G. Remark 2. In fact, Theorem 2 is stronger than the Hammersley–Clifford Theorem as it deals with arbitrarily factorization models, not necessarily ones induced by a graph. However, the proof for the two theorems are very similar, both based on the use of ˚ y -transform. Corollary 2. Let jXi j > 1 for all i 2 N . Then M D M if and only if D . Proof. The “if” part is obvious. We now prove the “only if” part. Suppose a 2 . For a given y 2 X , define P 2 RX C as follows. P .x1 ; : : : ; xn / D

2 1

if xi D yi for all i 2 a otherwise:

(3.15)

54


Clearly, P 2 M and hence by assumption P 2 M . Since jXi j > 1 for all i 2 N , there exists x 2 X such that xi ¤ yi for all i 2 N . Now: ˚ay ŒP .x/ D

Y

P .xb ; yb /.1/

janbj

(3.16)

ba

Y

jaj

D 2.1/

1.1/

janbj

(3.17)

ba; b¤; jaj

D 2.1/ :

(3.18)

y

Hence, ˚a ŒP ¤ 1 and a 2 N by Theorem 2. As such, N and N N . By symmetry, we can also prove that N N and hence the result follows. t u By choosing different y’s, there are many transforms one can use. A natural question is if there is any one particularly more natural than the others. In the following, we will propose the log-orthogonal transform which seems to be a more natural one and has very interesting properties. Definition 5 (Log-orthogonal transform). Let P 2 RX C . The log-orthogonal transform of P consists of a collection of component functions f˚a ŒP W a N g where: 0 ˚a ŒP .x/ D @

Y

1 ˚ay ŒP .x/A

1 jX j

:

(3.19)

y2X y

Since ˚a ŒP .x/ is the geometric mean of ˚a ŒP .x/, the log-orthogonal transform shares the same properties as the ˚ y -transform. In fact, all the previous results regarding ˚ y -transform remain valid for the log-orthogonal transform. Theorem 3 (Log-orthogonality). Assume anb is nonempty. Then for all q 2 RX , X

log ˚a ŒP .x/q.xb / D 0:

x2X

As a ¤ b if and only if anb is nonempty or bna is nonempty, X x2X

for all a ¤ b.

log ˚a ŒP .x/ log ˚b ŒP .x/ D 0

(3.20)


55

Proof. To simplify notation, we let p.x/ D log P .x/. Then: X

log ˚a ŒP .x/q.xb /

(3.21)

x2X

D

X

0 log @

x2X

D

Y

1 jX1 j ˚ay .x/A

q.xb /

(3.22)

y2X

X 1 X log ˚ay .x/q.xb / jX j x2X y2X

(3.23)

! Y 1 X .1/jancj log .P .xc ; yc // D q.xb / jX j x; y2X ca

! 1 X X jancj .1/ p.xc ; yc / q.xb / D jX j x; y2X ca D

XX 1 X .1/jancj p.xc ; yc /q.xb / jX j ca

(3.24)

(3.25)

(3.26)

x2X y2X

X 1 X .1/jancj jXc jjXc jp.xc /q.xb / jX j ca x2X X X jancj .1/ p.xc /q.xb / D

.a/

D

ca

D

X

x2X

.1/ja\bjjej

ea\b

X

.1/janbjjd j

X

p.xd ; xe /q.xb /;

(3.27) (3.28) (3.29)

x2X

d anb

where .a/ is due to (3.1). On the other hand, we have: X

p.xd ; xe /q.xb /

x2X

X

(3.30) p.xd ; xe /q.xb /

(3.31)

jX j X jXd jp.xe /q.xb / jXb jjXd j x

(3.32)

jX j X p.xe /q.xb /; jXb j x

(3.33)

D jXfb[d g j

xb[d

D

b

D

b

which is independent of d . The theorem then follows from Proposition 1.

t u

56


The above theorem proves that all the functions flog ˚a ŒP ; a 2 P.N /g are orthogonal to each other. This explains why the proposed transform is named as it is.

3.2 Functions Approximation Using Theorem 2, we can find the minimal factorization model that a positive function P is in. Now, suppose that the model is too complex to work with, and hence one might aim to approximate P with one from a simpler model. The question is how to find such an approximation. To answer the question, we first need a measure to quantify the “quality” of an approximation. One commonly used measure is the generalized divergence distance. Definition 6 (Divergence distance). Let P; Q 2 RX 0 . The divergence distance D.P jjQ/ is defined as: D.P jjQ/ ,

X x2X

P .x/ log

X X P .x/ C Q.x/ P .x/; Q.x/ x2X

x2X

where we adopt the convention that 0 log.0=˛/ D 0 for ˛ 0 and ˛ log.˛=0/ D C1 for all ˛ > 0. Denote the support of any P in RX 0 (i.e., the set fx 2 X W P .x/ > 0g) by .P /. Then: (P P P .x/ x2.P / P .x/ log Q.x/ C x2X .Q.x/ P .x// if .P / .Q/ D.P jjQ/ D C1 otherwise. When P and Q are both probability distributions, then D.P jjQ/ is reduced to the Kullback–Leibler distance. Note also that while D.P jjQ/ is not continuous with respect to both P and Q, it is indeed a continuous function of Q for any fixed P . Now, if we fix the model that we desire the approximation to be in, then the function approximation problem is to choose a function Q in the given model that approximates P and minimizes D.P jjQ/. As we shall see, finding the optimal Q is equivalent to solving an I-projection problem. In [14], Dykstra and Lemke gave a necessary and sufficient condition for the approximation to be optimal. The optimal approximation can be found by the generalized iterative scaling algorithm [8, 9]. Some disadvantages of such an iterative approach are that it needs intensive computation power, and that the algorithm does not stop naturally. To overcome these difficulties, a computationally efficient suboptimal approximation is proposed. RecallQthat M is the set of positive functions in RX C that assume the form P .x/ D a2 Qa .xa /. Define log M D fq 2 RX W 9Q 2 M such that q.x/ D log Q.x/g:


57

Fig. 3.2 Approximating P X It can be checked easily that log M P is a vector subspace of R that contains functions p.x/ assuming the form a2 qa .xa /. In the original function approximation problem, we want to find a function Q 2 M such that it is nearest P in the divergence sense (see Fig. 3.2). The idea of our suboptimal approximation algorithm is very simple: replace the divergence distance measure with the `2 -norm jj log Q log P jj2 . There are two steps in our suboptimal approximation algorithm. Q The first step is to find an initial approximation Q.x/ of P which minimizes the `2 -norm. The second step is to further improve the approximation obtained from the first step.

Proposition 2 (First approximation). Given any P 2 RX C and a factorization Q Q , a2 ˚a ŒP .xa /. Then: model M , let Q.x/ QQ D arg min jj log Q log P jj2 : Q2M

In other words, QQ is the unique function in M such that log QQ is closest to log P in the `2 -norm. Q Proof. First, QQ 2 M if and only if log P Q 2 log M , which is a vector subspace of RX that assume the form p.x/ D a2 qa .xa /. To prove the proposition, it is sufficient to prove thatQlog P logQQ is orthogonal to all functions inP the vector space log M . Since P D a2P.N / ˚a ŒP , we have log P log QQ D b62 log ˚b ŒP . Note that if b 62 , then b 6 a for all a 2 . By Theorem 3, log ˚b ŒP is orthogonal to qa .xa / for all q 2 RX . Hence, log P log QQ is orthogonal to qa .xa / for all a 2 . The result then follows. t u By Proposition 2, QQ is selected as the first approximation of P . Now, we will describe how to improve the approximation. The simplest way to improve an approximation is by multiplying it by a suitable scaling factor. The following lemma gives the best scaling factor one should use. ˙ P; QQ 2 RX and c be a positive scalar. Lemma 2 (Second approximation). Let P P C Q is minimized if and only if c D Q Then D.P jjc Q/ x2X P .x/= x2X Q.x/.

58


Q Differentiating f with respect to c, we have Proof. Let fP .c/ , D.P jjcP Q/. P 0 00 2 Q f .c/ D x2X Q.x/ x2X P .x/=c and f .c/ D x2X P .x/=c . It 00 0 is to see that f .c/ > 0 and f .c/ D 0 when c D P P thus straightforward Q t u x2X P .x/= x2X Q.x/. Hence, the lemma follows. Q RemarkP 3. Suppose that P P is a probability function. By Lemma 2, multiplying Q Q by c D x2X P .x/= x2X Q.x/ simply normalizes QQ to a probability function. The following theorem gives bounds on the divergence distance from an approximation to the original distribution. ˙ P; QQ 2 RX and Q.x/ D c Q.x/ Q Theorem 4 (Error Bound). Let where c D C P P Q P .x/= Q.x/. Then: x2X x2X ˇˇ ˇˇ D.P jjQ/ jjP Qjj1 ˇˇlog P log QQ ˇˇ1 X ˇˇ ˇˇ 2 P .x/ ˇˇlog P log QQ ˇˇ2 :

(3.34) (3.35)

x2X

Proof. Note that: D.P jjQ/ D

X

P .x/ log

X X P .x/ Q.x/ P .x/ C Q.x/ x2X x2X

(3.36)

P .x/ log

P .x/ Q.x/

(3.37)

x2X

D

X x2X

D

X x2X

.a/

X x2X

D

X x2X

X x2X

P X Q P .x/ Q.x/ C P .x/ log P .x/ log Px2X Q Q.x/ x2X P .x/

P Q P .x/ Q.x/ P .x/ X Q C P x2X P .x/ log Q.x/ log Q Q P .x/ Q.x/ Q.x/ x2X

.P .x/ Q.x// log

P .x/ Q Q.x/

ˇ ˇ ˇ P .x/ ˇˇ ˇ jP .x/ Q.x/j ˇlog ˇ Q Q.x/

ˇˇ ˇˇ jjP Qjj1 ˇˇlog P log QQ ˇˇ1 .b/

2

(3.38)

x2X

X x2X

ˇˇ ˇˇ P .x/ ˇˇlog P log QQ ˇˇ2 ;

(3.39)

x2X

(3.40)

(3.41) (3.42) (3.43)

where (a) follows from the log-sum inequality ˇˇ the following ˇˇ fact P [24] and (b) from that jjP Qjj1 jjP jj1 C jjQjj1 D 2 x2X P .x/ and ˇˇlog P log QQ ˇˇ1 ˇˇ ˇˇ ˇˇlog P log QQ ˇˇ . t u 2


59

3.3 Model Selection So far, we have proposed a suboptimal algorithm to approximate a positive function P with one from a given factorization model. In this section, we will assume that the factorization model is not given, and that the approximation algorithm itself needs to determine which one is the “right model.” Certainly, model selection can be done naively using two steps: first, for each possible factorization model, find an approximation in that model that is closest to P . The second step is to search among all those approximations to determine which one is the most simplest but is still “good enough” to explain P . The disadvantage of this naive approach is, however, the complexity involved. The number of factorization models is extremely large, and hence it is nearly impossible to find the best approximation in every model. In this section, we will propose an automatic model selection algorithm that aims to return the “simplest” factorization model without incurring excessive approximation errors. Model Selection Algorithm Let P 2 RX C and error level be the maximum acceptable approximation error. 1. Find ˚a .P / for all a N and calculate its `2 -norms jj log ˚a .P /jj2 . 2. Set D P.N / and 2 D 0. 3. Choose jj log ˚a .P /jj2 is the smallest. q a 2 max./ such that P 2 2 4. If 2. C jj log ˚a .P /jj2 / x2X P .x/ error level, then set D nfag and q D 2 C jj log ˚a .P /jj22 . Go back to step 3. 5. Otherwise, the algorithm stops and M is the desired factorization model. The idea behind the above algorithm is very simple: start with a relatively complex factorization model M . The algorithm will then check if one can use a simpler model without introducing excessive errors in approximation. Toward this end, one chooses a 2 max./ such that jj log ˚a .P /jj2 is the smallest. Choosing a in max./ ensures that Mnfag is simpler than M , and requiring jj log ˚a .P /jj2 to be the smallest ensures that the resulting simplification incurs the smallest additional error. Note that we use the error bound in Theorem 4 to determine whether the simpler model would incur an unacceptably large approximation error or not. The same procedure is repeated until any further simplification will cause exceedingly large error. Q When the algorithm stops, QQ D a2 ˚a .P / gives the initial approximation of P . By P appropriate Pscaling, we can further improve the approximation to Q Q Q D Q. P .x/= x2X x2X Q.x//. By Theorem 4, the approximation error is upper bounded by: 0 1 s X X 2 @ jj log ˚a .P /jj22 A P .x/: a62N

x2X

60


4 Divergence Minimization Problem In the previous section, we considered a simple statistical inference problem, where the observed data are “complete” such that one can derive an empirical distribution from the data. We also assume that the empirical distribution is a sufficient statistic for estimating the underlying distribution. The main problem is to find an approximation Q in some factorization models that is “closest to” the empirical distribution P . However, observed data can be incomplete in many cases, making it impossible to derive an empirical distribution from the data. This section addresses the issue by formulating the problem as a divergence minimization problem. Let PX be the set of all probability distributions of X . We will treat it as a subset of RX 0 , elements of which are jX j 1 column vectors. 1 Lemma 3 (Limits in divergence distance). Let fQj g1 j D1 and fPj gj D1 be two sequences of probability mass functions in PX with pointwise limits Q0 and P0 , respectively. Then:

(i) lim infj !1 D.Qj jjPj / D.Q0 jjP0 /. (ii) for any P 2 PX , if .Qj / .P / for j sufficiently large, then: lim D.Qj jjP / D D.Q0 jjP /:

j !1

Proof. To prove (i ), first note that for each x 2 X , if either Q0 .x/ or P0 .x/ is positive, then: lim inf Qj .x/ log j !1

Qj .x/ Q0 .x/ Q0 .x/ log : Pj .x/ P0 .x/

Suppose now that Q0 .x/ D P0 .x/ D 0. Since Qj .x/ log we have: lim inf Qj .x/ log j !1

Qj .x/ Pj .x/

Qj .x/ Pj .x/,

Qj .x/ lim inf Qj .x/ Pj .x/ j !1 Pj .x/ D Q0 .x/ P0 .x/ D0 D Q0 .x/ log Q .x/

Q0 .x/ : P0 .x/

0 .x/ Therefore, lim infj !1 Qj .x/ log Pjj .x/ Q0 .x/ log Q for all x. Summing over P0 .x/ all x 2 X , we proved (i ). Then (i i ) follows from the continuity of the logarithmic function and that Qj .x/ logŒQj .x/=P .x/ < 1 for all x when j is sufficiently large. t u


61

˙ U be a positive function in RX , and Definition 7 (log-affine model [5, 6, 19]). Let C R be an r jX j matrix. A log-affine model M.R; U / is a subset of RX C containing | all Q’s in RX C such that Q.x/ D U.x/ exp.a RŒ; x/ for some column vectors a. Its closure M.R; U / is referred to as the extended log-affine model. In this chapter, we will assume that the first row of R is an “all-one” vector. Hence, if Q 2 M.R; U / (or M.R; U /), the function cQ is still in M.R; U / (or M.R; U / respectively), for all c > 0. In other words, M.R; U / and M.R; U / are closed under scaling with a positive real number. Suppose Q 2 M.R; U /. By definition, there exists a vector a such that log Q.x/ log U.x/ D a| RŒ; x. Let hRi be the vector subspace spanned by rows in R. Then Q 2 M.R; U / if and only if log Q> log U > 2 hRi. For any factorization model M , the set flog Q > W Q 2 M g is a vector subspace of RX . Consequently, M is also a log-affine model (where U is the all-one function).

4.1 I-Projection Problem Let C be a nonempty subset of PX and P 2 PX . The I-projection [11] of P on C is a probability mass function Q in C, which lies closest to P in the divergence distance, i.e., Q D arg minQ2C D.QjjP /. To simplify notation, we denote the minimal value as D.CjjP / and the minimum Q as arg D.CjjP /. For many applications, the set C is affine – there exists an r jX j matrix R and an r 1 vector r such that C D fQ 2 RX 0 W RQ D rg. For simplicity, we denote the subset as C.R; r/ and arg minQ2C.R;r/ D.QjjP / as arg D.R; rjjU /. An I-projection problem can be interpreted as a statistical inference problem: Let P be the prior distribution of a random variable and C be the set of distributions that are “consistent” with the observed experimental outcomes. A natural question is: Among all distributions in C, which the “best” estimate is. As we shall see in Sect. 5, a reasonable choice is to pick the one that is closest to the prior distribution. By doing so, the problem thus becomes an I-projection problem. In [12], an iterative algorithm called generalized iterative TK1 scaling algorithm was proposed to find I-projections. Suppose that C D kD0 Ck . Let Qj C1 D arg minQ2Ck D.QjjQj /, where k D j mod K. Since Ck involves less constraints, the I-projection is relatively easier to solve than the original one. In [12], it was proved that the limit of the sequence of I-projections is indeed exactly the I-projection of P on C. Proposition 3 (I-projection and log-affine model). Suppose D.R; rjjU / < 1. Then Q D arg D.R; rjjU / if and only if U.x/ exp.a| RŒ; x/ if x 2 S (3.44) Q.x/ D 0 otherwise ; where (i) RŒ; x denotes the x t h column of the matrix R, (ii) a is an r 1 column S vector, and (iii) S , Q2C.R;r/ W .Q/.U / .Q/.

62


Furthermore, for any q 2 C.R; r/, we have D.qjjU / D D.qjjQ/ C D.QjjU /, where Q D arg D.R; rjjU /. t u

Proof. See [10].

Corollary 3. Let U and P be two positive probability mass functions such that P .x/ D U.x/ exp.b| RŒ; x/ for an r 1 column vector b, i.e., P .x/ 2 M.R; U /. Then arg D.R; rjjP / D arg D.R; rjjU /. Proof. The corollary follows from that D.QjjP / D D.QjjU / b> RQ and that b> RQ is a constant for all Q 2 C.R; r/. t u ˙ frj g1 Lemma 4 (Continuity). Let j D1 be a convergent sequence of probability mass functions with limit r0 such that C.R; rj / is nonempty and D.R; rj jjU / < 1 for all j . Let Qj , arg D.R; rj jjU /. Then limj !1 Qj D Q0 . Proof. As PX is compact, to prove the lemma, it suffices to show that if limj !1 Qj exists, then the limit is Q0 . Let the limit be Q . It is easy to show that Q 2 C.R; r0 /. Let qj D Qj C Q .x/

minx2.Q / Qj .x/ .Q0 Q / for j 1. Then qj is in C.R; rj / and limj !1 qj D Q0 . By Lemma 3, limj !1 D.qj jjU / D D.Q0 jjU / and limj !1 D.Qj jj U / D D.Q jjU /. As D.qj jjU / D.Qj jjU / by definition, we have D.Q0 jjU / D.Q jjU /. Finally, as D.qjjU / is strictly convex, it implies that Q D Q0 and the result follows. t u ˙ C be a closed and convex subset of PX . Then Proposition 4 (Continuity). Let arg D.CjjP / is continuous at P if D.CjjP / < 1. Proof. Suppose that D.CjjP0 / < 1 and Q D arg D.CjjP0 /. Then, the support .Q0 / is a subset of the support .P0 /, and hence there is an open ball B containing P0 such that .P0 / .P / for all P 2 PX \ B. Therefore, D.CjjP / < 1 for all P 2 PX \ B. By the strict convexity of D.QjjP /, arg D.CjjP / is uniquely defined over P 2 PX \ B. Let fPj g1 j D1 be a sequence of distributions such that limj !1 Pj D P0 . Let Qj D arg D.CjjPj /. By the compactness of PX , to prove the proposition, it suffices to show that if limj !1 Qj exists, then the limit is Q0 . Suppose the limit exists and is equal to Q . Then: D.Q0 jjP0 / D.Q jjP0 / .a/

(3.45)

lim inf D.Qj jjPj /

(3.46)

lim sup D.Qj jjPj /

(3.47)

j !1

j !1

.b/

lim sup D.Q0 jjPj / j !1

.c/

D D.Q0 jjP0 /;

(3.48) (3.49)


63

where (a) follows from Lemma 3, (b) from D.Qj jjPj / D.Q0 jjPj /, and (c) from the continuity of D.QjjP / for a fixed Q. Consequently, we have D.Q0 jjP0 / D D.Q jjP0 /. As D.CjjP0 / < 1, by the strict convexity of D.QjjP /, we have Q D Q0 and the proposition follows. t u ˙ U be a positive function. Proposition 5 (I-projection and Log-affine model). Let Then Q D arg D.R; rjjU / if and only if Q 2 C.R; r/ \ M.R; U /. Proof. We first begin with the “only if” part. Suppose Q D arg D.R; rjjU /. By definition, Q 2 C.R; r/. To prove Q 2 M.R; U /, it suffices to construct a sequence in M.R; U / such that its limit is equal to Q. Suppose fqj g1 j D1 is a convergent sequence of positive probability mass functions with limit Q. Let rj DRqj and Qj D arg D.R; rj jjU /. By Lemma 4, limj !1 Qj DQ. As qj 2 C.R; rj / and qj is positive, Qj is positive by Proposition 3 and therefore belongs to M.R; U / for all positive integers j . Hence, Q 2 C \ M.R; U /. To prove the “if” part, first notice the existence of a sequence of functions fqj g1 j D1 in M.R; U / such that limj !1 qj ! Q. Since M.R; U / is closed under scaling, we can assume that qj 2 PX \ M.R; U /. Let rj , Rqj . By Proposition 3, qj D arg D.R; rj jjU /. By Lemma 4, we have Q D limj !1 qj D arg D.R; rjjU / and the proposition follows. t u

4.2 Divergence Minimization and Iterative Minimization Let C0 ; C1 ; : : : ; CK1 be nonempty, closed, and convex subsets of PX , M be a nonempty subset of PX , and .P / be a continuous nonpositive function.1 The sets C0 ; C1 ; : : : ; CK1 are called pseudo-constraints; M is a statistical model, and .P / is a function about the prior probability of P . The larger the value of .P / is, the more probable the underlying model is P . Definition 8. Let mk 0 for k D 0; : : : ; K 1. A divergence minimization (DM) problem is an optimization problem of the following form:

P Minimize .P / D K1 kD0 mk D.Ck jjP / .P / subject to P 2 M and P 2 PX :

(3.50)

As we shall see, the formulation of the problem is not arbitrary but is directly linked to real-world statistical inference problems. Without going through the details, we interpret a DM problem as follows (see Fig. 3.3). Each pseudo-constraint Ck corresponds to a piece of evidence about the underlying probability mass function of a random variable X . How well a probability 1 Generally speaking, it is not important whether .P / is nonpositive. We impose such a condition merely to ensure that the fitness/distance measure .P / is always nonnegative.

64


Fig. 3.3 Divergence minimization

distribution P explains such an evidence Ck is measured by D.Ck jjP /. The weighting coefficient mk is introduced to reflect the “importance or credibility” of the evidence Ck . The function .P / corresponds to “our prior knowledge” about thePunderlying probability distribution. Roughly speaking, the function .P / D K1 kD0 mk D.Ck jjP / .P / measures how well P explains the evidences fC0 ; C1 ; ; CK1 g (subject to the credibility weightings and prior knowledge of .P /). The DM problem thus aims to find a probability distribution in a model M that best explains all the evidences. For example, let X be the outcome of a biased dice. Suppose that P one has thrown the dice n times and has determined its average outcome r D 6iD1 iP .i /, where P is the empirical distribution from the n throws of the dice. In such a case, P Ck D fP 2 PX W 6iD1 iP .i / D rg. Clearly, the larger the value of n is, the more important the evidence is. In the following, we will study the properties of a DM problem. ˙ C be closed and convex. Then D.CjjP / is a convex Lemma 5 (Convexity). Let function of P . Hence, if .P / is concave, then .P / is a convex function of P . Proof. Let P0 ; P1 2 PX and Qi D arg D.CjjPi / for i D 0; 1. Then for any 0

1, we have: D.Cjj.1 /P0 C P1 / D..1 /Q0 C Q1 jj.1 /P0 C P1 / .a/

.1 /D.Q0 jjP0 / C D.Q1 jjP1 /

D .1 /D.CjjP0 / C D.CjjP1 /; where (a) follows from the convexity of D.QjjP /.

t u

In the following, we propose an iterative algorithm to solve the DM problem. The first step is to rewrite the DM problem as follows: Minimize .P; Q0 ; ; QK1 / D subject to k Q 2 Ck ; for k D 0; : : : ; K 1 P 2 M:

PK1 kD0

mk D.Qk jjP / .P / (3.51)


65

Problem (3.51) is equivalent to the DM problem: P is a minimum of the DM Problem, and Qk is the I-projection of P on Ck for k D 0; 1; : : : ; K 1 if and only if .P ; Q0 ; ; QK1 / is a minimum in (3.51). Therefore, the DM problem can be solved by finding the minimum of (3.51). We propose a greedy algorithm that solves (3.51). Iterative Minimization (IM) Algorithm Step 1: Start with j D 1 and an initial guess P1 2 PX \ M. Step 2: For k D 0; 1; : : : ; K 1, let Qjk , arg D.Ck jjPj /. P P K1 K1 k Step 3: Let Qj D kD0 mk Qj = kD0 mk . Note that Qj 2 PX as Ck is a subset of PX . P Step 4: Let Pj C1 D arg minP 2PX \M . K1 kD0 mk /D.Qj jjP / .P /. Step 5: j WD j C 1; Go back to Step 2. Q 0 ; : : : ; K1 W PX ! PX as follows: Define mappings ; ; k .P / D arg D.Ck jjP / for k D 0; : : : ; K 1; Q /D .P

K1 X kD0

mk PK1 kD0

mk

k .P /;

K1 X

.P / D arg min

PO 2PX \M

(3.52) (3.53)

! Q /jjPO / .PO /: mk D. .P

(3.54)

kD0

Clearly, the IM algorithm returns a sequence of estimated probability distributions Pj such that Pj C1 D .Pj /. Furthermore, if .P0 / < 1, then by Q / are continuous at P0 . Proposition 4, k .P / and .P ˙ fPj g1 be a sequence of estimates generated Theorem 5 (Monotonicity). Let j D1 from the IM algorithm. Then the sequence f .Pj /g1 j D1 is monotonic decreasing. In other words, .Pj / .Pj C1 /. Proof. Let Qj0 ; : : : ; QjK1 2 PX be fixed. For any P 2 PX , we have: .P; Qj0 ; : : : ; QjK1 / D

K1 X

" mk

D " D

Qjk .x/ log

x2X

kD0

"

X

X K1 X

x2X

K1 X kD0

#

P .x/

.P /

#

mk Qjk .x/ log P .x/

x2X kD0

X

Qjk .x/

!

.P / C C 0 #

mk Qj .x/ log P .x/ .P / C C 0

66

T.H. Chan and R.W. Yeung K1 X

D

!" mk

kD0 K1 X

D

!

# Qj .x/ Qj .x/ log .P / C C P .x/ x2X X

mk D.Qj jjP / .P / C C;

kD0

where C 0 and C are constants depending only on Qj0 ; : : : ; QjK1 . Hence, ! K1 X mk D.Qj jjP / .P / arg min .P; Qj1 ; : : : ; Qjk / D arg min P 2PX \M

P 2PX \M

and Pj C1 D arg minP 2PX \M .

PK1

.Pj C1 /

kD0

K1 X

kD0

mk /D.Qj jjP / .P /. As a result:

mk D.Qjk jjPj C1 / .Pj C1 /

kD0

D .Pj C1 ; Qj0 ; : : : ; QjK1/ .Pj ; Qj0 ; : : : ; QjK1 / D .Pj /; t u

and the theorem follows.

Since .Pj / is bounded below, f .Pj /g1 j D1 will converge. In addition, as fPj g1 is contained in a compact set, there exists at least one convergent j D1 subsequence of fPj g1 . Suppose P is the limit of a convergent subsequence j D1 fPjl g1 . In the next theorem, we will show that P is in fact a fixed point of jl D1 the IM algorithm, i.e., .P / D P Theorem 6 (Fixed-point theorem). For any given P , let P1 DP and Pj C1 D .Pj / for all j 2. Suppose P is the limit of a convergent subsequence fPjl g1 lD1 of fPj g1 j D1 . Assume that ! K1 X mk D.QjjP / .P / arg min P 2PX \M

kD0

is uniquely defined and is a continuous function of Q. Then P is a fixed point in the sense that P D .P /. P Proof. First, by the continuity of arg minP 2PX \M . K1 kD0 mk /D.QjjP / .P /, .P / is a continuous function of P . Hence, f .Pjl /g1 lD1 is also a convergent subsequence with limit .P / and .a/

.b/

.c/

.P / D lim .Pjl / D lim . .Pjl // D . .P //; l!1

l!1

(3.55)


67

where (a) and (c) follows from the continuity of and (b) follows from that Q /. Note that: limj !1 .Pj / exists. Let Q D .P .P / D

K1 X

! mk D.Q jjP / .P / C C

kD0

and

.P / D arg min

P 2PX \M

K1 X

! mk D.Q jjP / .P / C C;

kD0

where C is a constant. By the uniqueness assumption of the minimum of P . K1 kD0 mk /D.QjjP / .P / and .P / D . .P //, we have P D .P / and the result follows. t u Proposition 6 (Continuity). Let M be the extended log-affine model M.R; U /. Then for any Q 2 PX : arg min D.QjjP / D arg D.R; RQjjU /: P 2PX \M

Furthermore, arg minP 2PX \M D.QjjP / is a continuous function of Q. Proof. Let Q0 D arg D.R; RQjjU /. By Proposition 5, Q0 2 M.R; U /. Also, by the assumption that the entries in the first row of R are all one, Q0 is also in PX . We now prove that Q0 D arg minP 2PX \M.R;U / D.QjjP /. Let q 2 PX \ M.R; U /. Then by Corollary 3, Q0 D arg D.R; RQjjq/. Hence, D.Qjjq/ D D.QjjQ0 / C D.Q0 jjq/ D.QjjQ0 /. Consider any q0 2 PX \ M.R; U /. By definition, there exists a sequence fqj g1 j D1 in PX \ M.R; U / such that limj !1 qj D q0 . As D.QjjQ0 / D.Qjjqj /, we have D.QjjQ0 / limj !1 D.Qjjqj / D D.Qjjq0 /. Therefore, Q0 D arg minP 2PX \M.R;U / D.QjjP /. Furthermore, D.QjjQ0 / D D.Qjjq0 / if and only limj !1 D.Q0 jjqj / D 0. In other words, limj !1 qj D Q0 , or equivalently, q0 D Q0 . Therefore, Q0 is indeed the unique probability mass function which minimizes D.QjjP / for P 2 PX \ M.R; U /. Finally, arg minP 2PX \M.R;U / D.QjjP / being a continuous function of Q follows from Lemma 4. t u Corollary 4. Suppose .P / D 0, the set M is the extended log-affine model M.R; U /, and fPj g1 j D1 is a sequence of estimates constructed by the IM algorithm. Then every limit of a convergent subsequence of fPj g1 j D1 is a fixed point of the IM algorithm.

4.3 J-Projection Recall the generalized iterative scaling algorithm in Sect. 4.1. It is proved that the I-projection of P on an affine pseudo-constraint C D C0 \ C1 \ : : : \ CK1 can be obtained by alternately updating an estimate by “projecting” it onto one of the

68


K pseudo-constraints Ck . In practical situations, the pseudo-constraints are usually estimated from noisy data. As a result, it is possible that the obtained intersection of C0 \ C1 : : : \ CK1 is empty. In that case, the I-projection of P on C is not well defined. Moreover, if the generalized iterative scaling algorithm is implemented, it simply does not converge. To address this issue, we “extend” the I-projection as follows. Definition 9 (J-projection). Let C0 ; ; CK1 be K nonempty affine pseudoconstraints where Ck D fQ 2 RX be obtained by 0 W Rk Q D rk g. Let R P concatenating the rows of R1 ; ; Rk . In other words, R is a . K1 kD0 rk / jX j | | matrix defined as R| D ŒR0 ; ; RK1 . Then, we define the J-projection of U on fC0 ; ; CK1 g (with respect to given mk > 0 for k D 0; : : : K 1) as the minimum of the following DM problem: P Minimize .P / D K1 kD0 mk D.Ci jjP / (3.56) subject to P 2 M.R; U / \ PX : For simplicity, we assume without loss of generality that

PK1 kD0

mk D 1.

Theorem 7 (Extension). The J-projection problem is an extension of the Iprojection problem in the sense that if the intersection of C0 ; C1 ; ; CK1 is nonempty and U is positive,Tthen the J-projection of U on fC0 ; ; CK1 g is equal to the I-projection of U on K1 kD0 Ck . TK1 Proof. Let C D kD0 Ck and Q be the I-projection of U on C. By Proposition 5, P Q 2 C \ M.R; U /. As Q 2 M.R; U / and K1 kD0 D.Ck jjQ/ D 0, the probability mass function Q is the J-projection of U on fC0 ; ; CK1 g. Conversely, if Q is also the J-projection of U on fC0 ; ; CK1 g, then K1 X kD0

D.Ck jjQ / D

K1 X

D.Ck jjQ/ D 0;

kD0

which implies Q 2 C. By Proposition 5, Q is the I-projection of P on C.

t u

J-projection is advantageous over I-projection because it is well defined even when the intersection of C0 ; C1 ; ; CK1 is empty. Also, it is more flexible, having a freedom to adjust the parameters mk to reflect the “credibility or importance” of each individual pseudo-constraint. As such, it can be applied to more general scenarios. However, despite their differences, J-projection can still be found using the IM algorithm repeatedly. Specifically, the idea is as follows. First, we start with an estimate Pj , compute the I-projections of Pj on C0 ; ; CK1 , and denote them P k by Qj0 ; : : : ; QjK1 , respectively. Second, let Qj D K1 kD0 mk Qj . Then the updated estimate Pj C1 is chosen as one in PX \ M.R; U / which minimizes D.Qj jjP /. By Proposition 6, Pj C1 is the I-projection of U on C, where C D fQ 2 RX 0 W RQ D RQj g. This second step can be accomplished by the generalized iterative scaling algorithm.


69

5 Example: MLE In this section, we examine how divergence minimization can be used to solve an MLE problem.

5.1 MLE, EM Algorithm, and Divergence Minimization Let X and Y be discrete random variables with sample spaces X and Y, respectively. Suppose the outcome of Y is y . The incomplete-data maximum likelihood (ML) estimation problem is to find a probability mass function P .x; y/ in a given model P M such that the likelihood of y , denoted by x P .x; y / or more explicitly Pr.y I P /, is maximized. Expectation-Maximization (EM) algorithm is a popular technique to tackle ML estimation problem with incomplete data [20, 21]. The essence of the algorithm is to update the estimates of the underlying probability mass function by applying the following E-step and the M-step recursively.

EM Algorithm • E-step: Start with an estimate Pj . Suppose the conditional probability mass function of X given Y D y is Pj .xjy /. For each possible outcome x of X , let Lc .P / be the complete log-likelihood of .x; y / if the underlying probability mass function is P .x; y/, i.e., Lc .P / D log P .x; y /. Then the conditional expectation of the complete log-likelihood of .x; y / is given by: EPj Œlog Lc .P / D

X

Pj .xjy / log P .x; y /:

(3.57)

x2X

• M-step: P j C1 D arg minP 2M EPj Œlog Lc .P /. In the following, we will show that the above MLE problem is a DM problem, and algorithm is a special case of IM algorithm. Let C D fQ 2 PX Y W P that the EM Q.x; y / D 1g. In other words, C is the set of probability mass functions x2X such that Q.x; y/ D 0 if y ¤ y . Let P 2 PX Y and Q 2 C. Then: D.QjjP / D

X x;y

D

X x

Q.x; y/ log

Q.x; y/ P .x; y/

Q.x; y / log

Q.x; y / P .xjy/P .y /

(3.58)

(3.59)

70


D

X

Q.x; y / log

Q.x; y / log P .y / P .xjy/

(3.60)

Q.xjy / log

Q.xjy / log P .y / P .xjy/

(3.61)

x

D

X x

log P .y /;

(3.62)

and equality holds if and only if Q.x; y/ D

if y D y otherwise.

P .xjy/ 0

(3.63)

As a result, D.CjjP / D log P .y /, and hence the incomplete-data ML estimation problem can be formulated as a DM problem: Minimize .P / D D.CjjP / subject to P 2 M \ PX Y :

(3.64)

We can use the IM algorithm to solve the ML estimation problem. Since there is only one affine pseudo-constraint, the IM algorithm can be simplified as follows. First, start with an estimate Pj . Let Qj D arg D.CjjPj /, which is explicitly given by (3.63). Then the updated estimate Pj C1 is defined as arg minP 2PX Y \M D.Qj jjP /. Note that: D.Qj jjP / D

X

Qj .x; y/ log

Qj .x; y/ P .x; y/

(3.65)

Pj .xjy/ log

Pj .xjy/ P .x; y D y/

(3.66)

x;y

D

X x

D

X x

Pj .xjy/ log Pj .xjy/

X

Pj .xjy/ log P .x; y D y/:

(3.67)

x

As a result: arg

min

P 2PX Y \M

D.Qj jjP / D arg

max

P 2PX Y \M

X

Pj .xjy/ log P .x; y D y/:

x

Hence, the updating equation in the IM algorithm is exactly the same as in the EM algorithm. As the IM algorithm has a simple interpretation, the EM algorithm shares the same interpretation. A similar interpretation can also be found in [21].


71

The above divergence minimization approach can also be extended naturally to the maximum posterior probability (MAP) estimation problem. Suppose the underlying probability mass function is also a random variable. Let .P / D log Pr.P / be the logarithm of the prior probability of P . Then the logarithm of the posterior probability of P is proportional to .P / C log P .y /. Using a similar argument as above, the incomplete-data MAP estimation problem can be formulated as the following DM problem: Minimize .P / D D.CjjP / .P / subject to P 2 M \ PX Y :

(3.68)

5.2 Large-Scale MLE Let X be a discrete random variable with finite sample space X . To estimate the underlying probability mass function of X , K independent experiments are performed. In particular, for k D 0; 1; ; K 1, we perform mk independent trials of X in Experiment k. The outcomes of Experiment k are recorded in a table Tk , such that Tk .x/ is the number of occurrences of X that is equal to x in Experiment k. If all the entries in the tables Tk are known, MLE is simple. However, in some situations, the tables fTk W k D 0; 1; ; K 1g are only “partially accessible” due to physical constraint or technical difficulties in the experiments. In this chapter, we assume that only a linear function Fk .Tk /, instead of Tk , is known. Example 4. Let X be the outcome of a dice. Consider an experiment which consists of m independent trials of X . Suppose the empirical mean and second moment of the dice are observed. Hence, the function F.T / is known, where # " P6 T .x/ xD1 x m : F.T / D P6 2 T .x/ xD1 x m Subject to the incomplete information, the ML estimation problem is to find a probability mass function P 2 M such that the likelihood of F0 .T0 /; F1 .T1 /; : : : ; FK1 .TK1 / is maximized. In other words, we want to solve the following maximization problem: PK1 Maximize kD0 log Pr.fk I P / (3.69) subject to P 2 M; and P 2 PX ; where Pr.fk I P / is the likelihood of the event Fk .Tk / D fk , given that the underlying probability mass function is P . The above ML estimation problem can be solved in two steps: (1) find the loglikelihood function, and (2) minimize the log-likelihood. For simple functions Fk , the likelihood function can be computed exactly in a simple way. However, for

72


a general function Fk , the likelihood can be difficult to compute exactly, and an explicitly enumeration of all possible configurations of Tk is needed. This bruteforce approach can be computationally intensive, especially when the number of trials in an experiment is very large. To overcome this difficulty, a suboptimal approach is proposed by approximating the likelihood function with a simpler one. Then the suboptimal estimation is obtained by optimizing with respect to the simpler approximating function. The remaining question, however, is: how to approximate a log-likelihood function? For k D 0; 1; ; K 1, let Ck D fQ 2 PX W Fk .mk Q/ D fk g. Hence, Fk .Tk / D fk if and only if Qk 2 Ck , where Qk is the empirical probability mass function obtained in Experiment k, i.e., Qk .x/ D Tk .x/=mk . By Sanov’s Theorem [7], we have: 1 jX j log.mk C 1/ log Pr.fk I P / D.Ck jjP / C : mk mk Note that, for very large mk , the second term on the right-hand side of the above inequality is negligible. On the other hand, suppose q is a probability mass function such that q.x/ is a rational number with denominator mk . If in addition q 2 Ck , then: jX j log.mk C 1/ 1 log Pr.fk I P / D.qjjP / : mk mk Suppose mk is large, and hence we can assume that for any Q 2 Ck , there exists q 2 Ck that is closed enough2 to Q. In this sense, D.Ck jjP / is a fairly good approximation to m1k log Pr.fk I P /. Using this approximation, the ML estimation problem is simplified to the following DM problem: Minimize .P / D

PK1 kD0

mk D.Ck jjP /

subject to P 2 M; and P 2 PX :

(3.70)

As a final remark, it is worth to mention that the complexity in solving the above DM problem is independent of the sizes of m0 ; m1 ; ; mK1 . This makes the proposed approximation approach particularly useful when the sizes of experiments are large.

6 Conclusion This chapter studies a statistical inference problem using function factorization and divergence minimization. We began with the simplest scenario in which an empirical distribution can be derived from the observed experiment outputs and 2

Here, we say that q and Q are closed if jD.qjjP / D.QjjP /j is small.


73

is the sufficient statistics for the underlying probability distribution to be inferred. We proposed an explicit transform using which the minimal/simplest factorization model the distribution belonging to is found. Via the same transform, we can also approximate the distribution with ones from a simpler factorization model. When the data are incomplete (and hence impossible to derive the empirical distribution from the data), we formulate the inference problem as a divergence minimization problem. We proposed an iterative algorithm to solve the minimization problem and showed that the algorithm always converges. We established a close relationship between MLE problem and DM problem. In particular, we showed in one scenario that the MLE problem is equivalent to a divergence minimization problem, and that our proposed iterative minimization algorithm is the same as the well-known EM algorithm. In another scenario, we demonstrated that a MLE problem can be approximated by a DM problem. By doing so, the complexity of the MLE problem can be greatly reduced.

References 1. Besag, J.: Spatialinteraction and the statistical analysis of lattice systems. J. Royal Stat. Soc. B (Methodological) 36(2), 192–236 (1974) 2. Chan, T., Yeung, R.W.: New results in probabilistic modeling. Doctoral Thesis, The Chinese University of Hong Kong (2000) 3. Chan, T., Yeung, R.W.: On factorization of positive functions. Proc. 2001 IEEE Int. Symp. Inform. Theory, Washington DC, USA, pp. 44, July 2001 4. Chan, T., Yeung, R.W.: On maximum likelihood estimation and divergence minimization. Proc. 2002 IEEE Int. Symp. Inform. Theory, Lausanne, Switzerland, pp. 158, July 2002 5. Christensen, R.: Log-Linear Models. Springer, New York (1990) 6. Christensen, R.: Log-Linear Models and Logistic Regression. Springer, New York (1997) 7. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley Interscience, New York (1991) 8. Csiszár, I.: I-divegence geometry of probability mass functions and minimization problems. Ann. Probab. 3(1), 146–158 (1975) 9. Csiszár, I., Tusnady, G.: Information geometry and alternating minimization procedures. In: Dedewicz, E.F., et al. (eds.) Statistics and Decisions, pp. 205–237 (1984) 10. Csiszár, I.: Sanov property, generalized i-projection and a conditional limit theorem. Ann. Probab. 12(3), 768–793 (1984) 11. Csiszár, I.: A geometric interpretation of Darroch and Ratcliff’s generalized iterative scaling. Ann. Stat. 17(3), 1409–1413 (1989) 12. Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. Ann. Math. Stat. 43(5), 1470–1480 (1972) 13. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum-likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc. B. 39(1), 1–38 (1977) 14. Dykstra, R.L., Lemke, J.H.: Duality of I projections and maximum likelihood estimation of log-linear models under cone constraints. J. Am. Stat. Assoc. 83, 546–554 (1988) 15. Figueiredo, M.T., Leitao, J.M.N.: Bayesian estimation of ventricular contours in angiographic images. IEEE Trans. Med. Imag. 11, 416–429 (1992) 16. Frey, B.J.: Graphical Models for Machine Learning and Digital Communication. MIT, Cambridge, Mass (1998) 17. Ising, E.: Beitrag sur Theorie des Ferromagnetismus. Zeit. fur Physik 31, 253–258 (1925)

74 18. 19. 20. 21.


Jordan, M.I.: Learning in Graphical Models. Kluwer, Boston (1998) Lauritzen, S.L.: Graphical Models. Clarendon, Oxford (1996) McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (1997) Neal, R.M., Hinton, G.E.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 355–368. Kluwer, Boston (1999) 22. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, San Mateo, California (1988) 23. Redner, R., Walker, H.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26(2), 195–239 (1984) 24. Yeung, R.W.: A First Course in Information Theory. Kluwer/Plenum, New York (2002)

Chapter 4

Wave Localization on Complex Networks Richard Berkovits, Lukas Jahnke, and Jan W. Kantelhardt

Abstract In this chapter we consider the role played by Anderson localization in transport through complex networks, for example, an optical network. The network is described by a tight binding Hamiltonian, which may be used to determine the properties of the Anderson transition according to the statistical properties of its eigenvalues. The Anderson transition properties of different complex networks will be studied, emphasizing the role played by clustering on the localization of waves. We shall show that new complex topologies lead to novel physics, specifically clustering may lead to localization. Keywords Anderson localization • Complex and random graphs • Information theory MSC2000 Primary 05C80; Secondary 90B18, 90B15, 05C50, 05C82.

1 Introduction 1.1 Motivation Anderson localization continues to spur excitement although half a century has passed since it was first conceived in the context of electron transport through disordered metals [1, 2]. Since then, new systems in which this phenomenon occurs were suggested and verified, such as light in strongly scattering media [3, 4] or photonic crystals [5, 6], acoustical vibrations in glasses [7] or percolation systems

R. Berkovits () Minerva Center and Department of Physics, Bar-Ilan University, Ramat-Gan 52900, Israel e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 4, © Springer Science+Business Media, LLC 2011

75

76

R. Berkovits et al.

[8], and very recently atomic Bose–Einstein condensates in an aperiodic optical lattice [9,10]. The Anderson transition, first predicted in 1958 [1], has been a central topic in condensed matter physics for the last half century. The main idea is that a phase transition from extended (metallic) to localized (insulating) eigenstates exists as a function of the disorder or energy of a quantum system. This transition will be manifested by a change in the transport through the system from metallic to insulating conductance. The transition is conceptually different from the canonical explanation for the existence of insulators in solid state (i.e., the Fermi energy is in the gap between energy band), since for the Anderson insulator the Fermi energy is in the middle of the band. Nevertheless, since the states available are localized due to the disorder, no current can pass through the system. Although the insulating behavior is due to disorder, it is nevertheless quite different in origin from the percolation transition. While in the percolation transition no direct path exists between the two edges of the sample, in the Anderson transition such paths do exist, nevertheless the probability of current transversing the sample is exponentially small due to constructive interference between time reversed paths. Thus, the Anderson transition takes into account the quantum (or wave) nature of the particle diffusing through the sample, while the percolation transition is classical in nature. The fact that the Anderson transition depends on interference effects between time reversed paths leads to an interesting dependence of the Anderson transition on the dimensionality of the system. The lower critical dimension, below which the system is localized for all values of disorder, is believed to be two [11], since the probability of returning to the origin (i.e., constructive interference due to time reversal symmetry) stays nonzero in the limit of infinite system size for d 2. The upper critical dimension (for which the system remains metallic for any amount of disorder) remains uncertain although it is generally believed to be infinity [12–16]. Most of the studies concerning the Anderson transition have considered regular lattices with on-site or bond disorder. Interest in the influence of the unusual topologies of complex networks on the properties of quantum interference is rising. Indeed, recently the Anderson transition in particular networks, namely smallworld networks [17–19], the Cayley tree [20], random regular graphs, Erdös–Rényi graphs, and scale-free networks [21], was studied. One may justify this interest by the light the study of the Anderson transition on complex networks sheds on the general properties of the transition, but here we would like to emphasize that complex networks could be actually realized in real-world situations such as optical networks. Hence, we shall begin by describing an example of an experimentally realizable system of a complex network, i.e., an optical network. We shall show that optical networks may be described using a tight binding Hamiltonian. We shall then describe how one can determine the properties of the Anderson transition using our knowledge of the statistical properties of the eigenvalues of the Hamiltonian. Then the Anderson transition properties of different complex networks will be studied. Finally, we investigate the role played by clustering on the localization of waves. We shall show that new complex topologies lead to novel physics, specifically clustering may lead to localization.

4 Wave Localization on Complex Networks Fig. 4.1 Possible realizations of optical networks constructed from beam splitters (cavities) represented by circles and optical fibers (wave guides) represented by lines. (a) A regular lattice. (b) A complex network

a

77

b

1.2 Optical Network A fresh view on the relevance of complex network models to real-world localization comes from considering optical networks. There is a long history of considering optical (or microwave) systems as a tool to analyze localization behavior. In 1982, Shapiro [22] has generalized a model proposed by Anderson et al. [23] to describe localization in disordered systems. Instead of considering a tight-binding description of a lattice, Shapiro considers a model in which sites are represented by beam splitters or cavities and bonds by optical fibers or wave guides (see Fig. 4.1a). From a theoretical point of view, this description is convenient to describe the system using the scattering matrix formalism. This description was taken a step further by Edrei et al. [24] who applied it to describe the dynamics of wave propagation (e.g., acoustic or light waves) in a disordered medium. One should realize that such a description can form the basis of a realistic network which may be built in a laboratory. For example, a network of beam splitters and optical fibers similar to the one sketched in Fig. 4.1b may be constructed and its localization properties experimentally studied. Since optical fibers have a very low loss rate, there is no essential difference between connecting neighboring nodes of the network, or far away ones. Moreover, since in any realistic optical setup the length of the optical fiber is much longer than the wave length even for nearestneighbor nodes, there is no correlation between the phase gained by the wave as it transverses the distance between two nodes and the physical distance between the nodes. Thus, in principal any form of a complex network (small-world network, Cayley tree, random regular graph, Erdös–Rényi graph and scale-free network) can be constructed and measured on an optical bench. Although constructing such a small network seems experimentally feasible, to the best of our knowledge it has not been performed. Thus transitions in the transport properties of coherent waves on complex networks with long-range links are relevant to typical real-world communication networks [25, 26] and can be studied experimentally. Alternatively, one might consider a network of wave guides on the nanoscale similar to photonic lattices [5, 6]. Although we illustrated the analogy between optical and electronic systems using the scattering matrix formulation, it is nevertheless quite general. It is known that the scalar wave equation is a good approximation for the propagation of an optical

78

R. Berkovits et al.

wave in an inhomogeneous medium as long as polarization effects are not important. The scalar wave equation may be written as r 2 C

.r/ @2 D 0; c 2 @2 t

(4.1)

where .r/ D 1 C ı.r/ describes the local fluctuations in the dielectric constant and c is the speed of light. Assuming a monochromatic wave, one may rewrite .r; t/ D .r/ exp.i!t/, where ! is the frequency of the wave. Inserting .r; t/ into the scalar wave equation (4.1) will result in r2

ı.r/

D

! 2 c

:

(4.2)

When this is compared with the stationary Schrödinger equation with a varying potential U.r/ D U C ıU.r/, r2

C

2m ıU.r/ „2

D

2m .E U / ; „2

(4.3)

it can be seen that the Schrödinger equation and the scalar wave equation in random media are identical up to constants. Thus, one may use techniques developed in the field of electronic localization to study the properties of optical networks. One such approach is determining the Anderson transition from the statistics of the eigenvalues of the Hamiltonian describing the system. In the extended phase, the distribution of the eigenvalues is expected to follow the appropriate Gaussian ensemble (i.e., if the Hamiltonian follows time-reversal symmetry it should follow the Gaussian orthogonal ensemble, GOE, while in the presence of a magnetic field which breaks time-reversal symmetry it is expected to follow the Gaussian unitary ensemble, GUE). On the other hand, in the localized regime the eigenvalues are expected to follow Poissonian statistics. Thus, by changing the disorder a system described by (4.3) should exhibit a transition from the appropriate Gaussian ensemble in the metallic phase to a Poisson statistics in the localized phase. This transition should show the usual finite-size behavior expected for a second-order phase transition, and thus finite-size scaling may be used to determine the transition point [27].

1.3 Introduction Anderson Model and Level Statistics To study wave localization, we consider the Anderson Hamiltonian [1], H D

X i

i ai ai

X .i;j /

tj;i aj ai ;

(4.4)

4 Wave Localization on Complex Networks Fig. 4.2 The distribution P .s/ for a 500 sites scale-free graph with D 4 (m D 2). A clear transition from Wigner to Poisson is observed as a function of disorder

79 1 W=2 W=5 W=7 W=10 W=15 W=20 W=25 W=35 Wigner Poisson

0.8

P(s)

0.6 0.4 0.2 0

0

1

2

3

4

5

s

where the first part represents the disordered on-site (node) potential (homogeneous distribution W=2 < i < W=2) and the second part describes the transfer between each pair of nodes .i; j /. For optical waves, one has ti;j D exp.i'i;j / for connected nodes ('i;j is the optical phase accumulated along the bond), and ti;j D 0 for disconnected nodes. For simplicity, we restrict ti;j to random values ˙1; the Hamiltonian thus remains in the orthogonal symmetry class. The extension to unitary symmetry is straightforward. In this scenario, the on-site disorder W results from variations in the optical units (beam splitters) located at the nodes. Then we applied level statistics [28] to determine the localization behavior of the modes and to extract the quantum phase-transition points. The Hamiltonian is diagonalized, and one obtains N eigenvalues Ei (where N is the number of nodes in the graph). A very useful characterization of the statistics of the eigenvalues is the distribution P .s/ of adjacent level spacings s, where s D .Ei C1 Ei /=hEi C1 Ei i, and h: : :i denotes averaging over different realizations of disorder or other random features of the network such as different realizations of node connectivities. One expects the distribution to shift as function of disorder from the Wigner surmise distribution (characteristic of Gaussian orthogonal ensembles), Pw .s/ D

s s 2 ; exp 2 4

(4.5)

at weak disorder to a Poisson distribution (characteristic of localized states) at strong disorder, Pp .s/ D expŒs: (4.6) Examples of such transitions for a particular complex network can be seen in the following sections (Fig. 4.2). As the on-site disorder W increases, P .s/ shifts from the GOE toward the Poisson distribution. Additional features of the Anderson

80

R. Berkovits et al.

transition such as the fact that all curves intersects at s D 2 and the peak of the distribution climbs along the Poisson curve for larger values of W are also seen. At the transition, the distribution will not change as function of the system size, while for weaker disorder the distribution will become more GOE like as the system size increases and for stronger disorder it will become more Poisson like as function of the system size. The transition point can thus be determined by a finite-size scaling procedure [27], which will be described in detail further on.

2 Statistical Properties of the Spectra of Complex Networks In the following section, we shall present the results of numerical studies of the spectral statistics of different classes of complex networks which have been mainly published in [21]. The features of the Anderson metal-insulator transition are found to be similar for a wide range of different networks. A metal-insulator transition as a function of the disorder can be observed for different classes of complex networks for which the average connectivity is small. The critical index of the transition corresponds to the mean field expectation (i.e., D 0:5). When the connectivity is higher, the amount of disorder needed to reach a certain degree of localization is proportional to the average connectivity, though a precise transition cannot be identified. The absence of a clear transition at high connectivity is probably due to the very compact structure of the highly connected networks, resulting in a small diameter even for a large number of sites.

2.1 Characteristics of the Different Networks We shall begin by a short definition of the characteristics of the different networks we consider. 2.1.1 Random Graph A random graph (or random regular graph) is a graph with N nodes, each is connected exactly to k random neighbors [29]. The diameter of a graph is the maximal distance between any pair of its nodes. In a random graph, the diameter d is proportional to ln N . In Sect. 2.3, we shall present results of the level spacing distribution for random-regular graphs with k D 3. 2.1.2 Erdös–Rényi Graphs In their classical model from 1959, Erdo¨ s and Rényi (ER) [30] describe a graph with N nodes where every pair of nodes is connected with probability p resulting

4 Wave Localization on Complex Networks

81

in hki D Np. For a large random graph, the degree distribution follows the Poisson distribution: hkik : (4.7) kŠ The diameter of such a graph follows: d ln N , similar to a random graph. In Sect. 2.3, we have specifically calculated the level distribution for hki D 3; 3:1; 3:2; 3:5; 4; 5; 7:5, and 10. P .k/ D ehki

2.1.3 Scale-Free Networks Scale-free (SF) networks [31] are networks where the degree distribution (i.e., fraction of sites with k connections) decays as a power-law. The degree distribution is given by [32]: P .k/ D ck ; m < k < K; where c D . 1/m1 and K D mN 1=.1/ [33,34], is the power-law exponent, m is a lower cutoff, and K is an upper cutoff . Thus, there are no sites with degree below m or above K. The diameter of the SF network can be regarded as the mean distance of the sites from the site with the highest degree. For graphs with 2 < < 3, the distance behaves as d ln.ln.N // [35], and for D 3 as d ln.N /= ln.ln.N // [36]. This anomalous behavior stems from the structure of the network where a small core containing most of the high degree sites has a very small diameter. For higher values of , the distance behaves as in ER, i.e., d ln.N /. The hki of a SF graph is obtained by the following expression: hki D

1 K 2 m2 : 2 K 1 m1

(4.8)

For > 2 and large enough N , the average degree, hki, is a constant. The results for SF networks presented in Sect. 2.2 correspond to D 3:5; 4; 5 with m D 2 (lower cutoff), and D 4; 5 with m D 3. Due to their small diameter, SF networks with < 3 were not considered. 2.1.4 Double-Peaked Distributions In order to find hierarchical relation between the different graphs, we studied also some variations on these graphs. For a random graph, we changed the degree of a small percent of the nodes, so we have a graph with double-peaked distribution. Thus, the average connectivity, hki, is the average degree of the nodes. Several examples were taken: changing 5% of the nodes to k D 5 (instead of k D 3) resulting in hki D 3:1, or changing 5% of the nodes to k D 10 (hki D 3:35).

82

R. Berkovits et al.

Replacing 20% of the nodes connectivity for the previous cases will result in hki D 3:4 (for k D 5 nodes) and hki D 4:4 (for k D 10 nodes). Additionally, to relate with previous results of the metal insulator transition on a Cayley tree [20], we checked a tree in which 5% of its nodes have higher degree (k D 4) resulting in an average connectivity 3:05 and creating few closed trajectories – loops.

2.2 Method Now we turn to the calculation of the spectral statistics of these networks. First, one must construct the appropriate network structure, i.e., to determine which node is connected to which. This is achieved using the following algorithm [33, 34, 37]: 1. For each site, choose a degree from the required distribution. 2. Create a list in which each site is repeated as many times as its degree. 3. Choose randomly two sites from the list and connect this pair of site as long as they are different sites. 4. Remove the pair from the list. Return to 3. The diameter of a graph is calculated by building shells of sites [32]. The inner shell contains the node with the highest degree, the next contains all of its neighbors, and so on. Of course, each node is counted only once. The diameter of the system is then determined by the number of shells. Two more options that were considered are defining the diameter by the most highly populated shell, or by averaging over the shells. The diameter obtained by the various methods are quite similar. The energy spectrum is calculated using the usual tight-binding Hamiltonian defined in (4.4). The on-site energies, "i are uniformly distributed over the range W=2 "i W=2. The hopping matrix element tj;i is set to 1 for nearest neighbor nodes which are determined according to the network structure, and to 0 for unconnected nodes. We diagonalize the Hamiltonian exactly and obtain N eigenvalues Ei (where N is the number of nodes in the graph) and eigenvectors i . Then we calculate the distribution P .s/ of adjacent level spacings s defined above. An example for such a transition is presented in Fig. 4.2 where a scale-free graph with D 4 and m D 2 was considered. As W increases P .s/ shifts toward the Poisson distribution. Additional hallmark features of the Anderson transition such as the fact that all curves intersects at s D 2 and the peak of the distribution “climbs” along the Poisson curve for larger values of W are also apparent. Similar transition from Wigner to Poisson statistics is seen also for the other networks considered in this study. The transition point can be determined more accurately from calculating [27]: R1 D R 21 2

P .s/ds Pp .s/ds

R1 R2 1 2

Pw .s/ds Pw .s/ds

;

(4.9)

4 Wave Localization on Complex Networks 1

0.9

γ

Fig. 4.3 as function of W for different SF graphs sizes ( D 4 and m D 2). The typical behavior for finite size transition is seen, where a crossing in the size dependence of between the metallic (small values of W ) and localize (large value of W ) regime is seen

83

0.8 500 sites 1000 sites 2000 sites

0.7

0.6

5

10

15

20

25

30

W

where ! 0 as the distribution tends toward the Wigner distribution, and ! 1 if the distribution approaches the Poisson distribution. One expects that as the system size increases, the finite size corrections will become smaller resulting in a distribution closer to a Wigner distribution in the metallic regime and to Poisson in the localized one. At the transition point, the distribution should be independent of the system size. In Fig. 4.3, we plot the behavior of as function of W for several sizes of a scale-free graph. Indeed, decreases with system size for small values of W while it increases with size for large values of W . All curves should cross around a particular value of disorder signifying the critical disorder. From finite size scaling arguments [27], one expects that around the critical disorder will depend on the disorder and network size, L, in the following way: ˇ ˇ ˇW ˇ .W; L/ D .Wc ; L/ C C ˇˇ 1ˇˇ L1= ; W c

(4.10)

where C is a constant. This relation enables us to extract both the critical disorder Wc and the critical index . Scaling of the numerical data according to (4.10) yields two branches corresponding to the metallic and localized regimes, which are clearly seen in Fig. 4.4. The estimated values of and Wc (see Table 4.1) are extracted by fitting the branches to a fourth order polynomial.

2.3 Results The calculations for all networks mentioned above are performed for M different realizations, where M D 1;000; 400; 200; : : : ; 50 for the corresponding number of nodes: N D 200; 500; 1;000; : : : ; 4;000. Except for the Cayley-tree networks for which M D 4;000; 2;000; 1;000; : : : ; 125; 64 for the corresponding tree sizes:

84

R. Berkovits et al.

Fig. 4.4 The scaling of according to (4.10) for different SF, D 4; m D 2, networks sizes. Two branches, corresponding to the metallic and localized regimes, appear

1 0.8

γ

0.6 0.4 500 sites 1000 sites 2000 sites

0.2 0

0

10

20

30

40

50

/W−1|L1/ν

|Wc

Table 4.1 Networks showing the localization transition. The value of l is for N D 1;000 Network hki l Wc Scale-free, D 4, m D 2 2:97 12:46 15:7˙0:9 0:55˙0:11 Random-regular (RR) 3 11:8 11:9˙0:26 0:66˙0:08 Erdös–Rényi 3 9:45 20:5˙0:23 0:68˙0:08 Cayley-tree 3 10 11:44˙0:06 0:51˙0:045 Cayley-tree with loops 3:05 10 12:4˙0:1 0:54˙0:075 RR “double peak” 3:1 10:28 14:1˙0:3 0:85˙0:41

N D 63; 127; 255; : : : ; 2;047; 4;095 or L D 6; 7; 8; : : : ; 11; 12 (where L is the number of “generations” of the tree). Another exception is for Erdös–Rényi graphs in which hki is between 3 and 3:5. The low connectivity of the graphs results in one main cluster and relatively large number of not-connected nodes (about 5%). Thus, the calculations are made only for the largest cluster of each realization, since a procedure that considers all the nodes is skewed by the eigenvalues of small disconnected clusters [38]. A clear localization transition is observed for a group of graphs which are all characterized by an average degree hki smaller than 3:1, and an averaged last occupied shell l (for N D 1;000 sites) greater than or equal to 9:45. The results are summarized in Table 4.1. The results for all the graphs (including those which show no clear signs of transition) can be scaled according to their average degree hki. The higher the value of hki is, the higher is the value of W needed to obtain a specific value of . Thus, the higher the average degree, the more metallic the system is, which makes sense. A cross section at D 0:6 of all curves is shown in Fig. 4.5 as a function of hki. The hki of the networks studied in Fig. 4.5 as well as the averaged last occupied shell l for N D 1;000 sites are presented in Table 4.2.

4 Wave Localization on Complex Networks 60 50 40 W(γ= 0.6)

Fig. 4.5 The curves of the checked networks can be ordered by their hki. The values of their W . D 0:6/ are presented as a function of hki. One can notice the increasing W with hki

85

30 20 10 0

2

4

6

8

Table 4.2 The average connectivity hki of all the networks considered in this study, as well as the averaged last occupied shell l for N D 1;000 sites Network hki l (for N D 1;000) Scale-free, D 4, m D 2 Random-regular Random-regular “double peak” (p D 0:95 ! k D 3, p D 0:05 ! k D 5) Cayley-tree Cayley-tree with loops Erdös–Rényi Scale-free, D 3:5, m D 2 Erdös–Rényi Erdös–Rényi Random-regular “double peak” (p D 0:8 ! k D 3, p D 0:2 ! k D 5) Erdös–Rényi Random-regular “double peak” (p D 0:95 ! k D 3, p D 0:05 ! k D 10) Erdös–Rényi Scale-free, D 5, m D 3 Scale-free, D 4, m D 3 Random-regular “double peak” (p D 0:8 ! k D 3, p D 0:2 ! k D 10) Erdös–Rényi Erdös–Rényi Erdös–Rényi

2.97 3 3.1

12.46 11.8 10.28

3 3.05 3 3.28 3.1 3.2 3.4

10 10 9.45 9.36 9.31 9.03 9

3.5 3.35

8.33 7.99

4 4 4.5 4.4

7.51 7.37 6.05 6.13

5 7.5 10

6.31 5.02 4.1

10

86

R. Berkovits et al.

2.4 Observations The following observations can be gleaned out of the data for the different networks: (1) For all the networks that show a metal-insulator transition, is of order 1=2 except for the random-regular “double-peak” network which is the one with the highest value of connectivity that still shows a clear transition. A critical index of D 0:5 is expected for a system of infinite dimensionality. At hki D 3:1, the value of is significantly higher, but so is the estimate of the error bar. On the other hand, for the Erdös–Rényi graph with hki D 3:1, no clear transition is observed. (2) All networks with connectivity above 3:1 do not show clear signs of a metalinsulator transition. Nevertheless, one should be rather careful in interpreting this observation since, as is clear from Table 4.2, larger values of hki lead to smaller size, l, of the network for the same number of nodes. Moreover, from the two networks which have the same hki D 3:1, only the one with the higher value of l shows clear signs of the metal insulator transition. Thus, the absence of transition may be an artifact of the small size of networks with high average connectivity. (3) The critical disorder Wc fluctuates in the range of 12–20 (Table 4.2). Due to the small range of hki (2.97–3.1), it is hard to determine any relation between k and Wc . (4) On the other hand, there is a clear relation between the amount of disorder needed to reach a particular value of (i.e., the value of W needed to reach a certain degree of localization) and hki. As can be seen in Fig. 4.5, a linear dependence W . D 0:6/ / hki is observed.

2.5 Discussion Thus, the gross features of the Anderson metal-insulator transition are similar for a wide range of different networks. The critical index for all the networks studied here is within the range expected for a system of infinite dimensionality, and the connectivity influences the degree of localization. On the other hand, the fact that networks with high connectivity are very compact raises the problem of identifying the transition point. It is hard to extend the usual finite-size scaling method to networks with high connectivity since the number of sites grows very rapidly with size, while for small network sizes the crossover behavior of the curves is very noisy. This results in an inability to clearly identify the Anderson transition, although it cannot be ruled out the possibility that there is a critical connectivity for complex networks above which no transition exist. It is also worthwhile to add a general comment on the statistical method we have used to identify the transition. The Anderson transition is one of the best known examples of a quantum phase transition. Non-analyticities of the free energy


87

is the clearest sign for a classical phase transitions. One would expect that for quantum phase transitions, the free energy will be replaced by the ground state energy. Surprisingly this is not the case for the Anderson transition, since only the ground state wave function, not the ground state energy, exhibits a signature of the transition. Traditionally, numerically identifying the phase transition was performed by calculating the conductance or transmission through the system, which directly probes the properties of the wave functions. Later it has been realized by Shklovskii and his coworkers [27] that although the ground state energy does not contain information regarding the phase transition, statistical properties of the single-electron electrons encode such information. As we explained in the previous sections, the statistics of the energy spacings can be used to identify the transition and study its properties. Recently it was realized that information entropy can also be used as a tool to identify a quantum phase transition, even for the Anderson transition [39]. Constructing a density matrix of some part A of the system A by tracing out all other degrees of freedom, one may define the von Neumann entropy of the system SA D Tr. A ln A /. Averaging over different parts and realizations of disorder, one can use finite size scaling in a similar fashion to the one described for the spacing statistics to collapse the singular part of the entropy on two branches in a similar fashion to the one depicted in Fig. 4.4. Thus, the von Neumann entropy may be used as an additional method to identify the Anderson transition for complex networks and could be used to augment the statistical methods described in this chapter.

3 Complex Networks with High Clustering As introduced in the last sections, it is possible to study quantum phase transition by analyzing the spectrum of an Anderson model [1, 2] representing the complex network. The transition was obtained in Sect. 2 by introducing diagonal (on-site) disorder [21]. Alternative approaches would be non-diagonal (bond) disorder or percolation, i.e., removing some fraction of all sites or bonds. In this case, a classical transition [40] in which the infinite cluster breaks into finite pieces is found after the quantum phase transition [38]. Anderson and quantum percolation transitions, which seem to be in the same universality class, have been studied on different topologies including fractal structures [41], Cayley trees [20], and complex networks [21, 33, 34]. In all cases, the transitions were induced either by on-site disorder or by cutting bonds (percolation) and thus changing the degree distribution of the network [33, 34]. In this section, we want to show that it is possible to observe quantum and classical phase transitions by a mere topological change of the network. Even for zero disorder and unchanged degree distribution, we find a quantum phase transition by increasing the clustering of the network. Most of the results in this section have been published in [42].

88

R. Berkovits et al.

3.1 Clustering Complex networks have additional degrees of freedoms compared with standard lattices. The basic property of a network is its degree distribution P .k/ which defines the type of network. Some examples were discussed in Sect. 2.1. A property related to the degree distribution is the diameter d or the characteristic path length ` between two arbitrary nodes (small-world property) of the system. These are important global properties, but in real-world network one also observe local degrees of freedom [29, 43, 44]. Two of the most important ones are clustering and assortativity [45]. The latter describes the degree–degree correlation of the network. Although the assortativity is definitely an interesting subject, we will focus on the effect of clustering. To prevent interference between both, we keep the degree– degree correlation as low as possible. Clustering measures the cliquishness of a typical neighborhood and was introduced in the study of “small-world” networks [46]. Regular non-random networks with constant degree have high clustering and a large diameter comparable to standard lattices. When a fraction of the links is reconnected randomly, the diameter and the clustering drop. A completely random network has nearly zero clustering. On the other hand, real-world networks exhibit high clustering, indicating the presence of many loops on short length scales [29, 43, 44]. Quantitatively clustering can be defined as the probability of triadic closure, Ci D 2Ti =Œki .ki 1/ [46], where Ti is the number of triangles passing through vertex i and ki is its degree. Although clustering is a local quantity, a global measure C can be achieved by averaging over Ci . A global C cannot capture specific aspects of the network, e.g., varying degree–degree correlations can lead to networks with different topology but similar C [47–49]. On the other hand, the fully local Ci is hard to analyze. Therefore, it was suggested to average Ci within each degree class, yielding CN .k/ [26]. It is not possible to achieve all possible functional dependences of CN .k/ with k. To achieve high clustering of the higher degrees, the assortativity of the network has to be strong. When nodes with large degrees are connected to nodes with lower degrees, they cannot achieve a high clustering because the nodes with lower degrees do not have enough connections to participate in a large number of triangles. Depending on the strength of the degree–degree correlation, one finds an upper limit which can be approximated by CN .k/ D C0 .k 1/˛ ;

(4.11)

with ˛ between 1 for no and 0 for high assortativity. Because we want to study a phase transition induced by clustering, we want to keep the assortativity as low as possible. Therefore, we will restrict ourselves to ˛ D 1. An explicit linear dependence between C and C0 exists and variation of C0 between 0 (no clustering) and 1 (maximum clustering) is unambiguous. The connection between the global clustering coefficient C and C0 is a constant which only depends on


a

b

89

c

Fig. 4.6 Representative pictures of scale-free networks ( D 5) (a) without and (b,c) with clustering (C0 D 0:4; 0:6). All three networks have the same size of N D 250. The giant component has a size of (a) Ng D 250, (b) Ng D 203, and (c) Ng D 145. The actual global clustering coefficient is (a) C D 8 104 , (b) C D 0:34, and (c) C D 0:53. The global clustering coefficient of the giant component is (b) Cg D 0:10 and (c) Cg D 0:17. The logarithmically scaled gray scales presents the intensity of a mode with E 0:45, light gray indicating the highest, and black the lowest probability

P , C D C0 ./ with D K kDm .k 1/P .k/. It is possible to write using the basic parameters (m, K, , N ), using (4.8) and the paragraph following. We apply an algorithm suggested recently by Serrano and Boguña´ [50] to generate scale-free networks with tunable degree distribution P .k/ k (4.8) and CN .k/. In Fig. 4.6a–c, we show three representative pictures of scale-free networks as a result of the algorithm. They all have the same number of nodes with a rising C0 from left to right. One sees clearly that for higher clustering, more and more nodes disintegrate from the giant component. The reason is that for nodes with low degree, it is easier to achieve higher clustering in some small clusters. For example, nodes with a degree of two can achieve the highest clustering in triangles. In Fig. 4.7a, b, we show the degree distribution P .k/ and the degree-dependent clustering coefficient CN .k/, respectively. The lines are the theoretical values which were demanded. The circles correspond to results of the full network and the squares to results of the giant component. While the degree distribution is quite stable when going from the full system to the giant component, the clustering changes for all degrees drastically. It drops by at least a factor of two. A metal-insulator transition is well defined only on the giant component since the other clusters do not grow with system size. In the language of percolation, it is the infinite cluster because it becomes infinite for an infinite system size while the small clusters remain finite. Therefore, a localized state and an extended state are unambiguously defined only on the giant component. For the finite components, a localized state with a localization length larger than the system size cannot be distinguish from an extended state. Since the clustering of the giant component Cg differs from that of the full network C , it is a priori not clear which of them is a good order parameter for the

90

b

101

100 C(k)

P(k)

a

R. Berkovits et al.

10−2

10−2

10−5 100

10−1

101 k

102

100

101 k

102

Fig. 4.7 (a) Degree distribution P .k/ and (b) clustering coefficients CN .k/ for scale-free networks with D 4 [line in (a)], C0 D 0:65 [line in (b) according to (4.11)] and N D 15;000 nodes, averaged over 120 configurations. Circles for distributions regarding the whole network and squares for the giant component (with hN1 i D 11;906 nodes; shifted vertically by a factor of 2 for degree distribution)

transition. The ambiguity can be resolved by analyzing the functional dependence of C0 with the average clustering coefficient of the full network C and its giant component Cg . The illustrative results are shown in Fig. 4.8. Not only C is linearly dependent on C0 but also Cg if C0 0:9. We find no system size dependence which is important because we want to use a finite size scaling formalism to analyze the phase transition. Values for C0 0:85 should be treated with care because the algorithm fails to achieve such high clustering. Due to the linear dependence of all three versions of clustering coefficient, they are interchangeable. In the following, we will use the parameter C0 instead of C or CN .k/. The results shown in Fig. 4.8 are also a good measure for the quality of the algorithm. The black line is the theoretically achievable clustering with the given C0 . The algorithm does not achieve the theoretically possible clustering but is very near this value. But still a linear dependence holds for C and C0 , which is more important. In the inset, the gradient of the linear dependence is shown for different power-law exponent . One sees that for all , the algorithm does not achieve the theoretically possible clustering. Interestingly, for the clustering dependence of the giant component is nearly constant. We obtained similar, however, less reliable results when generating networks with the algorithm of Volz [51] fixing C instead of CN .k/. The problem of this algorithm is that it is not possible to control the assortativity. Therefore, when C is changed also the assortativity changes, making it difficult to judge whether both networks are comparable. Since each triangle represents a very short loop in the network, waves in networks with high clustering will have a high probability to return to the same node and to interfere. Since such interferences are the main reason for quantum localization, one may expect that strong clustering will induce localization.


91

1

0.8 Δ

0.8

0.6

0.6 0.4 0.2 3

C

4 λ

0.4

5

0.2

0 0

0.2

0.4

0.6

0.8

1

C0

Fig. 4.8 Test of the relation C D C0 ./ for a scale-free network with D 4 and m D 2. The black line is the ideal curve. The circles (squares) are results for the full network (giant component). The different gray scales represent different system sizes from N D 2;000 (black) to N D 20;000 (white). The dashed (dashed dotted) line is a linear fit for the full network (giant component). Inset: Results of the fits for ./. Line is the ideal curve. The circles (squares) are the results for the full network (giant component). The dashed line is the mean of for the giant component with hi D 0:34

3.2 Results To analyze the metal-insulator transition for networks with clustering, we use the formalism as introduced in Sect. 1.3. We study scale-free networks with various and C0 . The eigenvalues and eigenvectors are calculated by exact diagonalization of the Hamiltonian (4.4). Figures 4.6a–c show the intensities corresponding to three eigenmodes. For all three cases, the state is extended on the whole giant component and not present on the finite clusters. To extract the critical clustering C0;q , the critical disorder Wc , and critical exponent , we have to extend (4.10) to also include C0 . As in the case for the disorder W , we expect from finite-size scaling arguments [27] that around C0;q will behave exactly as around Wc : .C0 ; W; L/ D .C0;q ; Wc ; L/ C ŒR1 jC0 C0;q j C R2 jW Wc jL1= ;

(4.12)

where R1 and R2 are constants and L / ln .a.C0 /N / 1 . Using (4.12) we have determined C0;q and Wc for scale-free networks with various . We also checked that equivalent R 1results are obtained if other integral measures of P .s/ are studied, e.g., I0 D 12 0 s 2 P .s/ ds. 1 The N dependence is well established [35] but the Cc 0 dependence seems to be unexplored. Our data for N up to 105 suggest ln a / C0 C0;c . Since C0;q < C0;c , a and thus L depend weakly on C0 at the quantum transition.

92

R. Berkovits et al. 1 0.8 γ

0.8

0.6 0.4

0.6 P(s)

0.2

0.6

C

0.4

0.7

0.8

0

0.2 0

0

1

2

3

4

5

s

Fig. 4.9 Level spacing distribution P .s/ for optical modes on scale-free networks with D 5, N D 12;500 and no disorder, W D 0. A clear transition from Wigner (dashed curve) to Poisson (dash-dotted curve) behavior is observed as a function of the clustering coefficient prefactor that is increased from C0 D 0:0 (light gray curve nearest to dashed curve) to C0 D 0:90 (dark gray curve nearest to dashed dotted curve). Inset: localization parameter [see (4.9)] versus C0 for networks with N D 5;000 (black), N D 7;500 (very light gray), N D 10; 000 (light gray), N D12;500 (gray), and N D 15;000 (dark gray). A transition from extended modes for small C0 to localized modes for large C0 is observed at C0;q 0:69. The results are based on eigenvalues around jEj D 0:2 and 0:5 (taken from [42])

To convince the reader that clustering indeed induces a phase transition we show in Fig. 4.9 the level spacing distribution P .s/ of a large scale-free network without disorder (W D 0) but varied C0 , as well the two limiting cases PW .s/ and PP .s/. One can clearly see that the shape of P .s/ changes from Wigner to Poisson with increasing C0 . Comparing Figs. 4.9 with 4.2, one finds no deviation from the typical behavior known from disorder. We thus observe an Anderson-like transition although there is no disorder W and no changes in the degree distribution P .k/. The inset of Fig. 4.9 shows for five system sizes versus the clustering strength C0 . One can observe the quantum phase transition at the critical value C0;q 0:69 by the crossing of the five curves, indicating a system size-independent critical value of c 0:76. The phase diagram for the transition from localized (upper right) to extended (lower left) optical modes is shown in Fig. 4.10a. The horizontal axis (C0 D 0) corresponds to the case with no clustering studied before in Sect. 2.2, where the critical disorder Wc depends on . The main new finding of the present study regards the transitions on the vertical axis. Without disorder, the transition to the localized phase occurs at a critical clustering C0;q that depend on , i.e., the degree distribution. While even the strongest clustering C0 D 1 cannot achieve such a transition if < 4, values of C0;q < 1 are observed for > 4. The case D 4 seems to be limiting: this is the broadest degree distribution which allows a quantum phase transition upon increasing clustering.


a

93

b

1

0.8 0.6 ν

0.8

0.4 0.2

C0,q

0.6

0

c

0.5

1

5 λ

6

0.4 C0

0.8 0.2

0.6 0.4

0 0

5

10 Wq

15

20

0.2

4

Fig. 4.10 (a) Phase diagram for transitions from localized optical modes (upper right) to extended modes in parts of the spectrum (lower left) for different degree distribution exponents , D 4 (diamonds), 4.25 (circles), and 5 (squares). Data for C0 > 0:9 are not reliable for network generation reasons, and the error bar for the point at C0 D 1 is about 0.1. (b) Exponent for different and C0;q . The values are, within the error bars (not shown), consistent with the meanfield prediction D 0:5. (c) Quantum transitions without disorder (dark gray circles) and classical transitions (squares) as a function of the degree exponent . In the regime 4 < < 4:5 only quantum transitions occur. For W > 0 the curves move downwards making quantum transitions possible for < 4 (light gray circles for W D 5 and gray circles for W D 10). The figures are taken from [42]

If variations of C0 and W are considered, the full phase diagram can be explored. Evidently, smaller values of C0 are sufficient for quantum phase transitions if W > 0. Within our error bars, the critical exponent corresponds to the meanfield value D 0:5 for infinite dimensions (see Fig. 4.10b) as expected from the Anderson transition [16]. We obtained similar phase diagrams for networks with homogenous or Erdös–Rényi-type degree distributions (not shown). There are some similarities between the phase transition induced by clustering and quantum percolation. In both cases, the giant component becomes smaller for larger values of the order parameters. We also find a classical phase transition after the quantum phase transition. The relation between both types of transitions is still an open question. The changes in the degree distribution of the giant component are not sufficient to explain the classical transition. To make sure that the quantum transition is induced by clustering and not by a classical phase transition, we determine the corresponding classical critical clustering coefficient C0;c . For that we have analyzed the size N2 of the second largest cluster in the system which should increase with C0 if the giant component exists (C0 < C0;c ) and decrease for higher values of C0 if it broke down (C0 > C0;c ) [33, 34]. We find no indications of a classical transition for < 4:5, i.e., the giant component is not broken. For D 5, we find C0;c 0:85, significantly larger than

94

R. Berkovits et al.

C0;q 0:69 (see insets of Figs. 4.9 and 4.10c). We thus conclude that the quantum transition for W D 0 is clearly different from the classical one in two ways: (a) there is no classical transition between 4 < < 4:5 although a quantum transition is clearly seen, and (b) for > 4:5, the quantum transition occurs for lower C0 values than the classical one. This leaves an intermediate regime (C0;q < C0 < C0;c 1) in which all modes are localized although there is a spanning giant cluster.

4 Summary In summary, we have shown that quantum phase transitions of wave-like modes (similar to the Anderson transition and to the quantum percolation transition) can be obtained in a complex network without introducing on-site disorder or bond disorder or tampering with the degree distribution (i.e., the number and distribution of links). One only needs to change the clustering coefficient of the network, which corresponds to a rewiring procedure. We conclude that clustering represents a new degree of freedom that can be used to induce and study phase transitions in complex networks. Comparing systems with different clustering properties might enable one to find the most relevant cause of quantum localization. We propose that the phenomenon should be observable experimentally and relevant in complex coherent optical networks made of fibers and beam splitters. Such experiments will directly probe the influence of complex network topology on the Anderson localization of light [3–6].

References 1. Anderson, P.W.: Absence of diffusion in certain random lattices. Phys. Rev. 109, 1492–1505 (1958) 2. Kramer, B., MacKinnon, A.: Localization – theory and experiment. Rep. Prog. Phys. 56, 1496–1564 (1993) 3. Wiersma, D.S., Bartolini, P., Lagendijk, A., Righini, R.: Localization of light in disordered medium. Nature 390, 671–673 (1997) 4. Störzer, M., Gross, P., Aegerter, C.M., Maret, G.: Observation of the critical regime near Anderson localization of light. Phys. Rev. Lett. 96, 063904 (2006) 5. Schwartz, T., Bartal, G., Fishman, S., Segev, M.: Transport and Anderson localization in disordered two-dimensional photonic lattices. Nature 446, 52–55 (2007) 6. Lahini, Y., Avidan, A., Pozzi, F., Sorel, M., Morandotti, R., Christodoulides, D.N., Silberberg, Y.: Anderson localization and nonlinearity in one-dimensional disordered photonic lattices. Phys. Rev. Lett. 100, 013906 (2008) 7. Foret, M., Courtens, E., Vacher, R., Suck, J.B.: Scattering investigation of acoustic localization in fused silica. Phys. Rev. Lett. 77, 3831–3834 (1996) 8. Kantelhardt, J.W., Bunde, A., Schweitzer, L.: Extended fractons and localized phonons on percolation clusters. Phys. Rev. Lett. 81, 4907–4910 (1998) 9. Billy, J., Josse, V., Zuo, Z.C., Bernard, A., Hambrecht, B., Lugan, P., Clement, D., SanchezPalencia, L., Bouyer, P., Aspect, A.: Direct obeservation of Anderson localization of matter waves in a controlled disorder. Nature 453, 891 (2008)


95

10. Roati, G., D’Errico, C., Fallani, L., Fattori, M., Fort, C., Zaccanti, M., Modugno, G., Modugno, M., Inguscio, M.: Anderson localization of a non-interacting bose-einstein condensate. Nature 453, 895–898 (2008) 11. Abrahams, E., Anderson, P.W., Licciardello, D.C., Ramakrishnan, T.V.: Scaling theory of localization – absence of quantum diffusion in 2 dimensions. Phys. Rev. Lett. 42, 673–676 (1979) 12. Castellani, C., DiCastro, C., Peliti, L.: On the upper critical dimension in Anderson localization. J. Phys. A 19, 1099–1103 (1986) 13. Kunz, H., Souillard, B.: On the upper critical dimension and the critical exponents of the localization transition. J. Phys. Lett. 44, L503–L506 (1983) 14. Straley, J.P.: Conductivity near the localization threshold in the high-dimensionality limit. Phys. Rev. B 28, 5393 (1983) 15. Lukes, T.: Critical dimensionality in the Anderson-Mott transition. J. Phys. C 12, L797 (1979) 16. Efetov, K.B.: Anderson transition on a bethe lattice (the symplectic and orthogonal ensembles). Zh. Eksp. Teor. Fiz 93, 1125–1139 (1987) [Sov. Phys. JETP, 61, 606 (1985)] 17. Zhu, C.P., Xiong, S.-J.: Localization-delocalization transition of electron states in a disordered quantum small-world network. Phys. Rev. B 62, 14780 (2000) 18. Giraud, O., Georgeot, B., Shepelyansky, D.L.: Quantum computing of delocalization in smallworld networks. Phys. Rev. E 72, 036203 (2005) 19. Gong, L., Tong, P.: von Neumann entropy and localization-delocalization transition of electron states in quantum small-world networks. Phys. Rev. E 74, 056103 (2006) 20. Sade, M., Berkovits, R.: Localization transition on a cayley tree via spectral statistics. Phys. Rev. B 68, 193102 (2003) 21. Sade, M., Kalisky, T., Havlin, S., Berkovits, R.: Localization transition on complex networks via spectral statistics. Phys. Rev. E 72, 066123 (2005) 22. Shapiro, B.: Renormalization-group transformation for the Anderson transition. Phys. Rev. Lett. 48, 823–825 (1982) 23. Anderson, P.W., Thouless, D.J., Abrahams, E., Fisher, D.S.: New Method for a scaling theory of localization. Phys. Rev. B 22, 3519–3526 (1980) 24. Edrei, I., Kaveh, M., Shapiro, B.: Probability-distribution functions for transmission of waves through random-media – a new numerical-method. Phys. Rev. Lett. 62, 2120–2123 (1989) 25. Carmi, S., Havlin, S., Kirkpatrick, S., Shavitt, Y., Shir, E.: A model of internet topology using k-shell decomposition. PNAS 104, 11150–11154 (2007) 26. Vázquez, A., Pastor-Satorras, R., Vespignani, A.: Large-scale topological and dynamical properties of the internet. Phys. Rev. E 65, 066130 (2002) 27. Shklovskii, B.I., Shapiro, B., Sears, B.R., Lambrianides, P., Shore, H.B.: Statistics of spectra of disordered-systems near the metal-insulator-transition. Phys. Rev. B 47, 11487–11490 (1993) 28. Hofstetter, E., Schreiber, M.: Relation between energy-level statistics and phase transition and its application to the Anderson model. Phys. Rev. E 49, 14726 (1994) 29. For a recent review see: Albert, R., Barabási, A.L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74, 47–97 (2002) 30. Erdös, P., Rényi, A.: On random graphs. Publ. Math. Debrecen 6, 290–297 (1959) 31. Albert, R., Jeong, H., Barabási, A.L.: Error and attack tolerance of complex networks. Nature 406, 378–382 (2000) 32. Kalisky, T. , Cohen, R. , ben Avraham, D., Havlin, S.: Tomography and stability of complex networks. In: Ben-Naim, E., Frauenfelder, H., Toroczkai, Z. (eds.) Lecture Notes in Physics: Proceedings of the 23rd LANL-CNLS Conference, “Complex Networks”, Santa-Fe, 2003. Springer, Berlin (2004) 33. Cohen, R., Erez, K., ben Avraham, D., Havlin, S.: Resilience of the internet to random breakdowns. Phys. Rev. Lett. 85, 4626–4628 (2000) 34. Cohen, R., Erez, K., ben Avraham, D., Havlin, S.: Resilience of the internet to random breakdowns. Phys. Rev. Lett. 86, 3682 (2001) 35. Cohen, R., Havlin, S.: Scale-free networks are ultrasmall. Phys. Rev. Lett. 90, 058701 (2003)

96

R. Berkovits et al.

36. Bollobas, B., Riordan, O.: Mathematical results on scale-free random graphs. In: Bornholdt, S., Schuster, H.G. (eds.) Handboook of Graphs and Networks. Wiley-VCH, Berlin (2002) 37. Molloy, M., Reed, B.: The size of the giant component of a random graph with a given degree sequence. Combinator. Probab. Comput. 7, 295–305 (1998) 38. Berkovits, R., Avishai, Y.: Spectral statistics near the quantum percolation threshold. Phys. Rev. B 53, R16125–R16128 (1996) 39. Kopp, A., Jia, X., Chakravarty, S.: Replacing energy by von Neumann entropy in quantum phase transitions. Ann. Phys. 322, 1466–1476 (2007) 40. Lorenz, C.D., Ziff, R.M.: Precise Determination of the bond percolation thresholds and finitesize scaling corrections for the sc, fcc, and bcc lattices. Phys. Rev. E 57, 230–236 (1998) 41. Schreiber, M., Grussbach, H.: Dimensionality dependence of the metal-insulator transition in the Anderson model of localization. Phys. Rev. Lett. 76, 1687–1690 (1996) 42. Jahnke, L., Kantelhardt, J.W., Berkovits, R., Havlin, S.: Wave localization in complex networks with high clustering. Phys. Rev. Lett. 101, 175702 (2008) 43. Dorogovtsev, S.N., Mendes, J.F.F.: Evolution of Networks – From Biological Nets to the Internet and WWW. Oxford University Press, Oxford (2003) 44. Pastor-Satorras, R., Vespignani, A.: Evolution and Structure of the Internet: A Statistical Physics Approach. Cambridge University Press, Cambridge (2004) 45. Newman, M.E.J.: Assortative mixing in networks. Phys. Rev. Lett. 89, 208701 (2002) 46. Watts, D.J, Strogatz, S.H.: Collective dynamics of ’small-world’ networks. Nature 393, 440–442 (1998) 47. Serrano, M.A., Boguña´ , M.: Percolation and epidemic thresholds in clustered networks. Phys. Rev. Lett. 97, 088701 (2006) 48. Serrano, M.A., Boguña´ , M.: Clustering in complex networks. I. General formalism. Phys. Rev. E 74, 056114 (2006) 49. Serrano, M.A., Boguña´ , M.: Clustering in complex networks. II. Percolation properties. Phys. Rev. E 74, 056115 (2006) 50. Serrano, M.A., Boguña´ , M.: Tuning clustering in random networks with arbitrary degree distributions. Phys. Rev. E 72, 036133 (2005) 51. Volz, E.: Random networks with tunable degree distribution and clustering. Phys. Rev. E 70, 056115 (2004)

Chapter 5

Information-Theoretic Methods in Chemical Graph Theory Elena Konstantinova

Abstract During recent years, information theory has been used extensively in chemistry for describing chemical structures and providing good correlations between physicochemical and structural properties. In this chapter, we present a survey on information-theoretic methods which are used in chemical graph theory. Keywords Entropy • Information content of molecular graph • Informationtheoretic methods • Molecular graph • Shannon relation • Topological and information indices MSC2000 Primary 62B10; Secondary 92E10, 05C90, 94A17, 94A15.

1 Main Trends of Chemical Graph Theory Chemical graph theory is interested in the nature of molecular structure. All structural formulae of chemical compounds are molecular graphs, where vertices represent atoms and edges represent chemical bonds. Figure 5.1 gives the schematic representation of the derivation of a molecular graph from an alkane molecule. This graph is the hydrogen-suppressed one, which is the commonly used representation in chemical graph theory because hydrogen atoms are small and so add very little to the overall size of the molecule. Using the molecular graph, one can obtain, for example, the carbon-number index (the number of carbon atoms in the hydrocarbon molecule) which is known since 1844 as one of the first topological indices used E. Konstantinova () Department of Mathematics, Yeungnam University, 712-749 South Korea Sobolev Institute of Mathematics, Siberian Branch of Russian Academy of Sciences, 630090 Novosibirsk, Russia e-mail: e [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 5, © Springer Science+Business Media, LLC 2011

97

98

E. Konstantinova

C3 H8

Fig. 5.1 Schematic representation of the derivation of the molecular graph and the carbon-number index from an alkane molecule

Chemical formula ↓

H

H

H

H

C

C

C

H

H

H

H

Structural formula ↓

Molecular hydrogen–suppressed graph ↓

n=3 Topological index

in chemistry to characterize molecular structures. Basically, the topological index expresses in numerical form the two-dimensional topology of the chemical species it presents. Topological indices are designed by transforming a molecular graph into a number. They possess the remarkable ability of being able to correlate and predict a very wide spectrum of properties for a vast range of molecular species. The carbon-number index is well known to provide an effective measure of the molecular volume for the members of homologous series, and the molecular volume is known to be directly proportional to the carbon-number index [Rou89]. The graph-theoretic approach has been successfully used to illuminate different aspect of molecular structure and properties [Tri92]. It has found applications in chemical documentation [Mo65], calculation of quantum chemical parameters [RB79], isomer discrimination [BT77,Ra75], enumeration of constitutional isomers [Ba76], structure–property and structure–activity relationships [BC80, KH86]. The construction and investigation of topological indices that could uniquely represent the molecular topology is one of the main directions of chemical graph theory. Let us briefly present the most important trends in chemical graph theory where topological indices are commonly used.

5 Information-Theoretic Methods in Chemical Graph Theory

99

1.1 Characterization of Molecular Structures Any concept of molecular structure is a hypothetical sketch of the organization of molecules. Such a model object is a general theory and remains empirically untested. A model object has to be grafted onto a specific theory to generate a theoretical model which can be empirically tested. For example, when in 1878 it was suggested by Sylvester [Syl78] that the structural formula of a molecule is a special kind of a graph, it was an innovate general theory without any predictive potential. The main aim of his “algebro-chemical theory” was to apply the methods of classical invariant theory to the rapidly developing science of molecular chemistry. When the idea of combinatorics was applied on chemical graphs, it could be predicted that “there should be exactly two isomers of butane (C4 H10 )” because “there are exactly two tree graphs with four vertices” when one considers only the nonhydrogen atoms presented in (C4 H10 ) [Ba85]. This is a theoretical model of limited predictive potential. Although it predicts the existence of chemical species, given a set of molecules, e.g., isomers of hexane (C6 H14 ), the model is incapable of predicting any property. This is because of the fact that any empirical property maps a set of chemical structures into the set of real numbers and thereby orders the set empirically. Therefore, to predict the property from structure, we need a nonempirical ordering scheme which closely resembles the empirical ordering of structures as determined by an empirical property. This is a more specific theoretical model based on the same model object, that is a molecular graph, and can be accomplished using specific graph invariants or topological indices, where a graph invariant is a graph-theoretic property which is preserved by isomorphism [Ha69]. Mathematical characterization of a molecular graph may be also accomplished by its matrices. Among the different matrices used for the representation of chemical structures, the adjacency matrix has been most extensively used in chemistry. However, this matrix poses a serious problem in chemical documentation because as the size of the graph increases, they require a large number of operations for the testing of graph isomorphism. Another limitation of matrices is that they cannot be used as structural descriptors in the correlation or prediction of properties. Therefore, one of the cherished objectives in contemporary graph-theoretic research has been the discovery of a graph property, preferably a single numerical characteristic or a set of numbers derived from graphs, which would not only be easier to handle than the adjacency matrix itself but also uniquely related to molecular topology. Unfortunately, in spite of numerous attempts, attainment of this goal has remained elusive. In chronological order, Spialter [Spi63, Spi64a, Spi64b] was the first to undertake a search for a graph invariant which could uniquely characterize molecular topology. Spialter asserted that the characteristic polynomial of the adjacency matrix or atom connectivity matrix of a molecule is uniquely related to its topology. However, this notion was contradicted by later researchers who found that non-isomorphic graphs may possess identical characteristic polynomial [BX71]. These graphs are called co-spectral graphs [Tri92]. Later Randić [Ra79] conjectured that for trees the collections of distance degree sequences are sufficient

100

E. Konstantinova

to determine their isomorphism, but then it was reported that neither distance degree sequences nor path degree sequences could uniquely characterize the topology of trees [QS81, Sl82]. Some other graph invariants were also considered for unique characterization of molecules but for all of them counter-examples were found.

1.2 Isomer Discrimination Under above circumstances, there are two distinct ways in chemical graph-theoretic index research: (a) the simultaneous use of more than one index, i.e., a superindex to better characterize chemical structure as compared to a single numerical invariant [Bon83]; and (b) the development of indices with high discriminating power [Kon96, KV03]. Both of these ways are used for discriminating isomers associated with a particular empirical formula. An exhaustive analysis of the main results of isomer discrimination among alkanes, and polycyclic graphs embedded to the regular hexagonal, square, and trigonal lattices are presented in Sect. 4. Success in developing indices with more discriminating power is evident from the increasing popularity of different graph invariants in structure–property and structure–activity relationships.

1.3 Structure–Property and Structure–Activity Relationships A universally accepted paradigm is that similar molecules have similar properties. One of the earliest recognitions of this postulate can be traced to Fisher [F94] who modeled in 1894 drug–receptor interaction by a “lock and key” analogy. In other words, the similarity principle may be reformulated as follows: structures that differ little in the mathematical invariant properties will differ little also in their physical, chemical, and biological properties. The above immediately reveals a strategy to attack the problems of structure–property relationship. Rather than directly trying to relate a property to structure, we may instead investigate different graph invariants of a structure and then follow with property–property correlations in which relatively simpler mathematical properties are used to express more complicated or convoluted physicochemical and biological properties. A fair introduction to the molecular similarity can be found in the book by Johnson and Maggiora [JM90]. The earliest works on structure–activity relationships were done by Crum-Brown and Fraser [CBF68] in 1868–1869. Until the middle of the past century, most of the correlations found between biological activity of molecules and their structure were qualitative. Nowadays, the quantitative structure–activity relationship (QSAR) studies deal with classes of compounds of similar structures. The predictions made in such studies are valuable for the practical pharmacology, toxicology, industrial and environmental chemistry, and agrochemistry (for reviews, see books


101

[D96, HL95, K93]). The main approaches in QSAR developed independently for structurally similar molecules are: (1) additive models and (2) models based on physicochemical properties of molecules. In the additive models, biological activity is assumed to be an additive substituent property that varies within the class of structurally related compounds. The additive method technique developed by Free and Wilson [FW64] has found a wide application [BF69, FB71, H80, HGL90, K88]. One of the impressive results of this approach was the successful prediction of the butyrylcholinesterase (also known as BCHE or BuChE) inhibitory potency of 1-decyl-3-(N -ethyl-N -methyl-carbamoyl) piperidine hydrobromide 3 years before its synthesis (see [H77] for details). The DARC–PELCO method of Dubois [DLA73, DLA74] also belongs to the group of additive methods and makes use of formal structural parameters. It has a more reliable and better defined predictive power than the other addictive schemes. The basic assumption in the physicochemical models is that introduction of different substituents into the leading molecule alters its biological potency. The classical and widely used physicochemical model is that of Hansch [HSK72, H81]. The applications of this model are numerous (see [PBC73]). Compatibility of the Free-Wilson and Hansch methods was investigated in [K88, SHP81]. An extended physicochemical model of the molecular biological action consisting of topological, steric, electronic, and hydrophobic contributions was proposed by Mekenyan and Bonchev [MB86] as Optimized Approach based on Structural Indices Set (OASIS). One can find the comparative study of DARC/PELCO and OASIS methods in [MMB93]. In modern QSAR approaches, it is becoming quite common to use a wide set of theoretical molecular descriptors of different kinds. It has been estimated that more than 3,000 molecular descriptors are now available, and most of them have been summarized and explained [DB99, K00, TC00]. The most used methods are ADAPT [SBJ79], OASIS [MB86], CODESSA [KL94], MolConnZ [MC03], and DRAGON [TCM06]. One of the main advantages of using graph invariants in these investigations is explained by the fact that they can be calculated for any real or hypothetical molecular structure, whereas physicochemical parameters used in structure–property relationships or structure–activity relationships are not uniformly available [SBJ79]. It has also been pointed out of Randić [Ra84] that the non-uniqueness of graph invariants is not a very serious handicap for these relationships. For example, in the alkane series, properties like boiling point and octane number are not mutually well correlated and lie in different numerical scale. Therefore, a unique topological index, if discovered, cannot simultaneously correlate with both properties. On the contrary, if a graph-theoretic index shows excellent correlation with a specific property of a congeneric series, then this relationship can help to elucidate the structural (topological) origin of that property. During the last decades, a large number of papers and books were published on applications of graph invariants in predicting physical, pharmaceutical, and toxicological properties of different groups of molecules (see [BC80, Bon83, KH86, KR87, RB91, Tri92] for reviews). An excellent survey on the history of quantitative structure–activity relationships is presented in [S03].

102

E. Konstantinova

Some results on correlating ability of information indices in finding spectrum– structure relationships will be demonstrated on the several classes of organic and organometallic compounds in Sect. 5.

1.4 Design of Compounds of Desired Properties Structure–activity relationship is an explosive problem in drug design, where the development of a single drug can take anything from 8 to 20 years and involve a cost of some $100 millions. Until the middle of the past century, most of the correlations found between biological activity of molecules and their structure were qualitative. Nowadays, the quantitative drug design embraces all attempts to relate biological activity via mathematical models to other properties or characteristics of molecular structure. Most of the quantitative structure–activity relationships studies deal with classes of compounds of similar structures. The predictions made in such studies are valuable for the practical pharmacology whose efforts are to a great extent focussed on synthesizing and testing structural modifications of the leading compound, i.e., having an established biological activity. The current success rate in the testing of possible new drugs is only around one in 10,000. In drug design, one can synthesize a large number of derivatives from a “lead” structure. It is unusual that one to test 200,000 or more chemicals to discover a molecule that is marketable [CS78]. In many cases, one might be interested to know the property of a molecule not yet synthesized. Then the only solution is to estimate properties using theoretical parameters which can be calculated for any arbitrary chemical structure, real or hypothetical. Graph invariants have been widely used for the quantification of molecular similarity, and these molecular similarity measures are emerging as important tools in practical aspects of pharmaceutical drug design [BNV91, L90] and environmental risk assessment [BBG93, BG93, BG95]. Practical quantification of intermolecular similarity of two chemicals is dependent on the mode of their structural representation and the choice of the appropriate set of descriptors used for measuring similarity. In view of the fact that in many practical situations empirical properties for comparing molecules are not readily available, nonempirical parameters such as graph invariants are being used more frequently for selecting analogs and estimating properties. For example, the similarity method based on the graph invariants and presented in 1988 by Johnson, Basak, and Maggiora [JBM88] (see also [JM90]) was used by the Upjohn Company for drug screening and pharmaceutical design. In 1991, it was announced by the chemists at Upjohn Company that a new AIDS drug was discovered using this approach [RBT88]. More precisely, they discovered the BHAP series of compounds which are potent HIV–RT inhibitors and which were in their drug-development pipeline. It was a great applied success of the graph-theoretic methods. Thus, the main trends of chemical graph theory where topological indices are widely used are presented above. Now we concentrate our attention on the second and third trends and present results those being concerned with the application


103

of information theory to chemical graph theory. It is well known that application of some ideas from one scientific field to another one often gives a new view on the problems. Information, one of the most general ideas in contemporary science, should be expected to penetrate in various branches of science. In the past 30 years, it is applied extensively in chemical graph theory for describing chemical structures and providing good correlations between physicochemical and structural properties using information indices. Information theory was used in defining measures of the topological properties of molecules, namely, information of adjacency, incidence, polynomial coefficients of the adjacency matrix, and for distances of molecular graphs. It was shown by Bonchev and Trinajstić [BT77, BT78] that the latter has a greater ability for discrimination between structural isomers than all topological indices considered before and to be a very appropriate measure of molecular branching which is a critical parameter determining the relative magnitude of various molecular thermodynamic properties. Later they applied information theory to the problem of characterizing molecular structures and molecular topology [BT78, BKT79, BMT80, BMT81, BT82] by means of information indices [Bon83] which are just the quantitative measures of a given topological structure. The advantage of such kind of indices is in that they may be used directly as simple numerical descriptors in a comparison with physical, chemical, or biological parameters of molecules in quantitative structure–property and structure–activity relationships [BC80, KH76, KH86]. It can also be noted that information indices normally have greater discriminating power for isomers than the respective topological indices. The reasons for this are that information indices are not restricted to integral values as topological indices frequently are and information indices are formed from a summation of different magnitudes which is usually greater in number than that for the topological indices. This chapter is organized in the following way. First, the information theory approach is given and its application in chemical graph theory is shown in the chronological order. Then the information indices based on the distance in a graph are reviewed. For the described information indices based on the distance in a graph, their numerical results in discriminating tests on structural isomers and graphs are presented. Finally, the correlating ability of the information indices is demonstrated on the several classes of organic and organometallic compounds in finding spectrum–structure relationships.

2 Information-Theoretic Approach The science of information theory has grown mainly out of the pioneering studies of Shannon [Sh49], Ashby [As56], Brillouin [Bril56], and Kolmogorov [Kol69]. There is more than one version of information theory. In Shannon’s statistical information theory, information is measured as reduced uncertainty of the system. Ashby describes information as a measure of variety. In the algorithmic theory of Kolmogorov, the quantity of information is defined as the minimal length of a

104

E. Konstantinova

program which allows a one-to-one transformation of an object (set) into another. Applying information theory to material structures such as atoms, molecules, and crystals, as well as to different mathematical structures such as sets, groups, and graphs, the interpretation given in 1968 by Mowshovitz [Mo68a, Mo68b, Mo68c, Mo68d] is more appropriate. Let a given system I having n elements be regarded according to a certain equivalence relation, into k equivalence classes with cardinalities ni . Considering all the n elements partitioned into k classes, we can define the probability pi , i D 1; : : : ; k, for a randomly selected element of this system to be found in the i th class. Therefore, a finite probability scheme may be associated with the following structure: 0 1 1 2 3 ::: k @ n1 n2 n3 : : : nk A ; p1 p2 p3 : : : pk P Pk where n D i D1 ni ; pi D ni =n and kiD1 pi D 1. The information content I of a system with n elements is defined by Brillouin [Bril56] as follows: k X I D n log2 n ni log2 ni : (5.1) i D1

The logarithm is taken at basis two for measuring the information contents in bits. Another information measure is the mean information content of one element of the system I defined by means of the total information content or by the Shannon relation [Sh49]: k X I D I =n D pi log2 pi ; (5.2) i D1

where pi D ni =n. It is also called the binary entropy of a finite probability scheme. The application of information theory to different systems or structures is based on the possibility of constructing a finite probability scheme for every system. One can mention here that the criterium for partitioning the elements of a given system is not unique. Usually we deal with equivalence classes of events generated by grouping actual events and ignoring, at the same time, some unique properties of those events. For example, when V represents the vertex set of a chemical graph G, two methods of partitioning have been widely used: (a) coloring of G, where two vertices of the same color are considered equivalent, and (b) determination of the transitive sets or orbits of the automorphism group of G, whereafter vertices are considered equivalent if they belong to the same orbit. Excellent reviews are available on measures of complexity and computation of complexity parameters [Bon83, Mar71]. The number of information measures is equal to the number of ways in which a set of n elements may be partitioned into different subsets, that is, the number of Young diagrams for a given n. It is always possible to select for any system several


105

information measures, each of them closely connected with certain properties of the system. They reflect the essence of the idea of information, given by Ashby [As56] as a measure of the variety in a given system. This idea was used in graph theory and in chemical graph theory for characterizing graphs as well as molecular graphs and molecular structures. At first the information theory was applied to graphs in 1955 by Rashevsky [Rash55] who defined the so-called topological information content of the graph Itop . His definition is based on the partitioning of the vertices of a given graph into classes of equivalent vertices having the same valencies. In 1956, Trucco [Tr56a, Tr56b] made this definition more precise on the basis of an automorphism group of graphs. In the latter case, two vertices are considered equivalent if they belong to the same orbit of the automorphism group, i.e., if they can interchange preserving the adjacency of the graph. Later the topological information was used by Rashevsky [Rash60] in studying the possibility of self-generation of the life on earth. As for chemical structures, information theory has been successfully applied in the study of various molecular properties [BKK76, Mor55, Val63], in the field of molecular dynamics [BL72, BLB72] and quantum chemistry [AC72, AC74, DB74], in the description of the electronic structure of atoms [Bon79], in the interpretation of the Pauli principle and Hund rule [Bon81]. In particular, in quantum chemistry the information index of aromaticity was introduced by applying the Shannon relation to the density matrix of a molecule, and it was shown that this index reproduces well the classification of molecules as aromatic, nonaromatic, and antiaromatic [FBE80, FEP82]. Information indices of neighborhood symmetry such as structural information content (SIC ) and complementary information content (CIC ) developed by Basak with coworkers [BRG80, BM83] have been found as effective in predicting chemical properties of molecules. Later [BBG00, BGB04], it was shown that these indices were found to be minimally correlated with many other topological indices, and SIC is related to Brillouin’s [Bril56] measure of a system. For a review on these indices, see [B99, MH83]. A molecular topology determines a large number of molecular properties. It was found in the last years that some biological activities of molecules, and even carcinogenicity, are closely related to a molecular topology. Thus, it is of a pertinent interest for chemistry (as well as for other natural sciences) to have some quantitative measure reflecting the essential features of a given topological structure. As it was mentioned above, such measures are usually called topological indices in chemical graph theory. A lot of such indices have been suggested in the last 50 years [Ba83, DG77, DG98, En76, En97, Ho71, KH81, KR87, NTM95, Ra75, Ra84, Rou83, Ra91, Wi47, Wi48a]. Usually they have correlated more or less satisfactorily with molecular properties but could not discriminate well between structural isomers, often providing the same index for different isomers. One of the first topological indices reflecting the topological structure of a molecular graph was proposed by Wiener in 1947 [Wi47]. The Wiener number W was defined as the sum of all edges between all pairs of carbon atoms in hydrocarbons. Platt in [P52] recognized the importance of this graph invariant and tried to rationalize its interpretation. He has found that the Wiener number relates to the overall molecular volume.

106

E. Konstantinova

Several very successful linear regressions by Wiener involved yet another graph theoretical invariant P, the count of paths of length three. With these two invariants, Wiener obtained regressions for various molecular properties for several classes of compounds [Wi48a, Wi48b]. It was also shown that this number gives a good correlation with the thermodynamic properties of saturated hydrocarbon molecules but does not discriminate well among structural isomers. As it was mentioned at the end of Sect. 1, information indices are helpful tools for this goal.

3 Information Indices Based on the Distance in a Graph One can start looking for possible information indices among the graph invariants. Information indices are constructed for various matrices such as an adjacency matrix, an incidence matrix, a distance matrix, and a layer matrix, and also for some topological indices such as the Wiener number. In 1977, Bonchev and Trinajstić [BT77] introduced an information on distances to explain the molecular branching that is the critical parameter determining the relative magnitude of various molecular thermodynamic properties. Initially they used the information indices defined by Rashevsky for graphs. However, these indices are not suitable for describing branching properties of graphs since they cannot reflect the essence of branching. This may be exemplified by considering trees with five vertices presented in Fig. 5.2. The five vertices are partitioned in different orbits in the above three graphs: T1 .2; 2; 1/; T2 .2; 1; 1; 1/; T3 .4; 1/. From (1), the following values for the information content in bits are obtained: IT1 D7:61; IT2 D9:61; IT3 D3:61. One can see that this index cannot reproduce the obvious fact that the branching increases from a chain, through a branched tree, to a star. Hence, another approach to find an appropriate information measure of branching was used. One of the graph invariants is the distance matrix. Let G be a connected graph with the set of vertices V .G/, n D jV .G/j. The distance d.u; v/ between vertices u and v in a graph G is the length of the shortest path that connects these vertices. The distance matrix D Dk dij k; i; j D 1; : : : ; n; contains the distances dij D d.i; j / between the different pairs of connected vertices. Branching 5 1

2

3

4

T1 : {1,5}, {2,4}, {3}

5

1

2

3

3 4

2

5

4

1 T2 : {4,5}, {1}, {2}, {3} T3 : {1,2,3,4}, {5}

Fig. 5.2 Trees with five vertices and their orbits


107

is connected with the distance matrix in an obvious way, since with increasing branching the distances in the graph become smaller. This can easily be seen from the distance matrices of the trees T1 ; T2 ; T3 presented in Fig. 5.2: ˇ ˇ ˇ ˇ ˇ ˇ D.T1 / D ˇ ˇ ˇ ˇ ˇ

0 1 2 3 4

1 0 1 2 3

2 1 0 1 2

3 2 1 0 1

4 3 2 1 0

ˇ ˇ ˇ ˇ ˇ ˇ ˇ ˇ ˇ ˇ ˇ

ˇ ˇ ˇ ˇ ˇ ˇ D.T2 / D ˇ ˇ ˇ ˇ ˇ

0 1 2 3 3

1 0 1 2 2

2 1 0 1 1

3 2 1 0 2

ˇ 3 ˇˇ 2 ˇˇ ˇ 1ˇ ˇ 2ˇ ˇ 0ˇ

ˇ ˇ0 ˇ ˇ2 ˇ ˇ D.T3 / D ˇ 2 ˇ ˇ2 ˇ ˇ1

2 0 2 2 1

2 2 0 2 1

2 2 2 0 1

ˇ 1 ˇˇ 1 ˇˇ ˇ 1 ˇ: ˇ 1ˇ ˇ 0ˇ

As it was mentioned above, Wiener first made use of the connection between the distance matrix and branching defining the following topological index, which is called the Wiener number in chemical graph theory: W D

n 1 X dij : 2 i;j D1

(5.3)

However, the Wiener number often has the same value for different graphs. For reducing degeneracies, Bonchev and Trinajstić introduced an information ID on distances in a graph, considering all the matrix elements dij of the distance matrix D as elements of a finite probability scheme associated with the graph in question. Let the distance of a value i appears 2ni times in the distance matrix, where 1 i d.G/ and d.G/ D maxi;j 2V .G/ d.i; j / is the diameter of a graph . Then n2 matrix elements dij are partitioned into d.G/ C 1 groups, and d.G/ C 1 group contains n zeros which are the diagonal matrix elements. With each one of these d.G/ C 1 groups can be associated a certain probability for a randomly chosen distance dij to be in the i th group: 1 0 0 1 2 : : : d.G/ @ n 2n1 2n2 : : : 2nd.G/ A ; 1 p1 p2 : : : pd.G/ n where pi D 2ni =n2 and p0 D n=n2 D 1=n. According to (5.1) and (5.2), the information on distances of a given graph is defined as follows: I D n2 log2 n2 n log2 n

d.G/

X

2ni log2 2ni ;

(5.4)

i D1 d.G/

1 X 2ni 2ni 1 I D log2 log2 2 : n n n2 n

(5.5)

i D1

Since D is a symmetric matrix, one can consider, for simplicity of discussion, only the upper triangular sub-matrix that doespreserve all properties of the

108

E. Konstantinova

information measure. In that case, the following expressions for the mean and total information on distances are obtained: IDE D

d.G/

n.n 1/ X n.n 1/ ni log2 ni ; log2 2 2 i D1 d.G/

E

ID D

X i D1

2ni 2ni log2 ; n.n 1/ n.n 1/

(5.6)

(5.7)

is the total number of upper off-diagonal elements in the distance mawhere n.n1/ 2 trix D. These information indices correspond to the information on the distribution of distances in the graph according to their equality or non-equality and depend on the partitioning of the total number of distances into classes. From (5.7), we obtain the following information on distances for the three graphs with five vertices presented in Fig. 5.2: ˇ ˇ 1 ˇ T1 W ˇˇ 4 ˇ 4

2 3

10

3 10

3 2

2 10

4 1

1 10

ˇ ˇi ˇ ˇ ni ˇ ˇp i

ˇ ˇ i ˇ ˇ ni ˇ ˇp i

ˇ ˇ ˇ1 2ˇ i ˇ ˇ T3 W ˇˇ 4 6 ˇˇ ni ˇ 4 6 ˇ p i 10 10

I D .T2 / D 1:52

I D .T3 / D 0:97:

ˇ ˇ 1 ˇ T2 W ˇˇ 4 ˇ 4

10

E

2 4

4 10

3 2

2 10

E

I D .T1 / D 1:85 E

E

E

One can see that I D , as well as IDE D n.n1/ I D , reproduces the branching 2 properties of trees T1 ; T2 ; T3 decreasing regularity with increased branching. Moreover, Bonchev and Trinajstić have shown that IDE is a rather sensitive measure of branching, having different values for all trees with n D 4; 5; 6; 7; 8 (the total number of trees is 45). The number of all possible distributions of dij , i.e., number of different IDE , increases rapidly with the increase in the number of vertices in the graph. This makes IDE an appropriate quantity for distinguishing structural isomers. However, there is another possible information measure which can be defined on the basis of distances in the graph. Bonchev and Trinajstić introduced the information index IDW as the information on the realized distances in a given graph, which depends on the partitioning of the total distance. It is an information on the partitioning the Wiener number, which is the total distance of the graph, into groups of distances of the same absolute values. Since the Wiener number is given by Pd.G/ the formula W D i D1 i ni , and from (5.1) and (5.2), we obtain the following formulas: d.G/ X W ID D W log2 W i ni log2 i; (5.8) i D1 W

ID D

d.G/

X i D1

ni

i i log2 : W W

(5.9)


109

For the three five-vertices trees presented in Fig. 5.2, the following values are obtained: W

T1 W

IDW D 62:93; I D D 3:15;

T2 W

IDW D 57:55; I D D 3:20;

T3 W

IDW D 52:00; I D D 3:25:

W

W

It is easy to see that IDW decreases with branching. It is a more sensitive quantity than the Wiener number since it can distinguish two graphs having the same Wiener W number but different i and ni . It was checked that I D increases regularity with branching at lower values of n, but at higher ones some irregularity occurs and it cannot be used as a good measure of branching. As for IDW , it is a sensitive measure of branching having different values for all trees with n D 4; 5; 6; 7; 8. In [BT77], the values of both information measures IDE and IDW were inspected in comparison with several topological indices such as the Wiener number, the greatest eigenvalue of the characteristic polynomial 1 , the sum of the polynomial coefficients (or Hosoya index) [Ho71], the information on polynomial coefficients Ipc , and Randić connectivity index [Ra75]. The last index gives a very successful correlations with Carbon-13 NMR chemical shift sums in alkane series as it was shown in [RT88]. The inspection of these indices indicates the great sensitivity of the two information indices IDE , IDW to all structural details of the tree graphs. There are no two graphs among the 45 trees examined which have one and the same information on the graph distances. All the other listed indices are not so specific, and they often have the same value for different graphs. The same results E W were obtained for information indices I D and I D . Figure 5.3 presents the pair of trees having the same value of the Wiener number and the different values of IDE and IDW . The correlations between indices were also investigated and a very good correlations between both information measures, IDE , IDW , and the Wiener number W were found on this set of graphs. It was the first attempt to find out whether a certain common basis exists between the different topological indices for a better understanding of the essence of branching. Thus, the information measures introduced on the basis of distance matrix appear to be very appropriate indices for discrimination of graphs. The number of different values IDE for the graphs having the same number of vertices is limited by the number of all possible distributions n.n 1/=2 graph edges into k different groups. Since the number increases rapidly with increasing values of n, one may expect the information on graph distances to have a good ability of differentiation between structural isomers even for very large systems. It was one of the main result obtained by Bonchev and Trinajstić. It was confirmed later by Konstantinova, Paleev, and Diudea [KP90, Kon96, KD00] that the information-theoretic approach allows to design very sensitive

110

E. Konstantinova n=7 W = 48

IE D = 41.0774,

IW D = 203.0586

IE D = 40.9544,

IW D = 203.5488

IE D = 39.5676,

IW D = 195.5544

IE D = 61.8350,

IW D = 328.8834

IE D = 55.3448,

IW D = 312.3888

IE D = 51.2058,

IW D = 290.3661

W = 46

IE D = 39.9740,

IW D = 195.0642

n=8 W = 71

IE D = 62.0606,

IW D = 328.0287

W = 67

IE D = 58.7736,

IW D = 311.5341

W = 62

IE D = 52.3253,

IW D = 289.3857

Fig. 5.3 The pair of trees having the same value of the Wiener number and the different values of IDE and IDW

information indices based on the distance in a graph. The information distance index of vertex i was introduced in [KP90] and defined as follows: HD .i / D

n X dij dij log2 ; d.i / d.i / j D1

(5.10)


111

P where d.i / D nj D1 dij is the distance of a vertex i . Then the information distance index of graph vertices takes the following form: HDn

D

n X

HD .i /:

(5.11)

i D1

This is natural to extend considerations in deriving graph information indices from the distance matrix to other graph matrices. The above approach was applied to the layer matrix Dk ij k; i D 1; : : : ; n; j D 1; : : : ; d.G/, where ij is equal to the number of vertices located at a distance j from a vertex i , and d.G/ is the diameter of a graph. The information layer index of graph vertices is defined by the following formula: e.i / n n X X X ij ij H .i / D log2 ; (5.12) Hn D n n i D1 i D1 j D0 where e.i / D maxv2V .G/ d.i; v/ is the vertex eccentricity. It will be shown later that indices HDn and Hn have a great discriminating power among structural isomers. One more information index based on the distance matrix was considered by Skorobogatov, Konstantinova et al. [SK91] in structure–activity correlations. The information index H2 is defined by the following relation: H2 D

k X d.i /ki i D1

2W

log2

d.i /ki ; 2W

(5.13)

where ki ; i D 1; : : : ; k; is the number of vertices having the distance d.i /. This index gives the linear correlations with information mass-spectrum indices on the several classes of organic and organometallic compounds [NST93, NS93, NS96, SN93]. D’yachkov, Konstantinova, and Vilenkin (“On entropy and information of trees”, unpublished paper) consider the entropy HD , the marginal entropy HDi , and the information ID based on the distance matrix as follows: d.G/ n X n X dij dij 1 X ni i log2 i; HD log2 D 1 C log2 W 2W 2W W i D1 i D1 j D1

n 1 X d.i / d.i / log2 d.i /; (5.15) D 1 C log2 W 2W 2W 2W i D1 i D1 1 0 d.G/ n X X 1 @ ID 2HDi HD D 1 C log2 W d.i / log2 d.i / ni i log2 i A ; W i D1 i D1

HDi

n X d.i /

(5.14)

log2

(5.16) where ni is the number of vertex pairs being at a distance i from each other and Pd.G/ W D i D1 i ni .

112

E. Konstantinova

Let li be the number of matrix elements equal to i . The entropy H , the marginal j entropies H and Hi , and the information I are defined as follows: H

n d.G/ X X i D1 j D1

X ij ij 1 li i log2 i; log2 D log2 n.n 1/ n.n 1/ n.n 1/ n.n 1/ i D1 max

(5.17) j

H

d.G/

X

j D1

2nj 2nj 1 log2 D log2 n.n1/ n.n 1/ n.n 1/ n.n 1/

d.G/

X

2nj log2 2nj ;

j D1

(5.18) Hi n

n n log2 D log2 n; n.n 1/ n.n 1/

(5.19)

1 0 d.G/ max X X 1 @ I Hi C Hi H D log2 n 2nj log2 2nj li i log2 i A : n.n 1/ j D1 i D1 (5.20) W j The information indices I D ; HD , and H are based on the vector ni and the W

constants n and W that leads to their correlations. In particular, since I D D Pd.G/ log2 W W1 i D1 ni i log2 i , then from (5.14) we immediately obtain HD D W

I D C 1. Hence, it is enough to study the only index among them. All the above indices were considered in the discriminating tests among structural isomers.

4 Discriminating Isomers by Information Indices The discriminating power [BMT81, RCD85] is one of the basic characteristics of a topological index I and corresponds to a measure of its ability to distinguish among the non-isomorphic graphs, or the structural isomers, by distinct numerical values of index I . The theoretical evaluation of index sensitivity S on a fixed set M of non-isomorphic graphs can be achieved by the following formula: N NI ; (5.21) N where N D jM j is the number of graphs in a set M , and NI is the number of degeneracies of an index I within set M . By the definition, S D 1 means that among the elements of the set considered, no two non-isomorphic graphs have the same value of the index I . Initially, as it was shown in the previous section, the discriminating power of information and topological indices between 45 alkane trees was investigated by Bonchev and Trinajstić in [BT77]. Raychaudhary et al. [RR84] have continued these investigations on the set of 45 alkane trees as well SD


113

as on the set of 19 monocyclic graphs. They have defined the new information index known as graph distance complexity which has been found to have a high discriminating power. Razinger, Chretien, and Dubois [RCD85] explicitly pointed out the fact that the discriminating power of the Wiener index is very low in alkane series. The first discriminating tests among the polycyclic graphs were done by Konstantinova and Paleev [KP90] on the set of 1,020 subgraphs of the regular hexagonal and square lattices. Later, Konstantinova [Kon96] has tested information and topological indices for 2,562 subgraphs of the regular hexagonal lattice. Graphs of this class represent the molecular structures of unbranched catacondensed benzenoid hydrocarbons. The discriminating powers of topological and information indices as well as the Wiener polynomial derivatives were studied by Konstantinova and Diudea [KD00] on 3,006 subgraphs of the regular hexagonal lattice and on the set of 347 cycle-containing graphs with ten vertices and three- to eight-membered cycle. An exhaustive analysis of 12 information and topological indices based on the distance in a graph was performed by Konstantinova and Vidyuk [KV03] on 1,443,032 polycyclic graphs and 3,473,141 trees. The information indices ID , HDi , W I , H , HDn , Hn , H2 , I D presented in Sects. 2 and 3 and the topological indices such as the Wiener index, the Schultz index, the Balaban index, and the Randić index were examined in the discriminating tests. The Schultz molecular topological index [Gu94] is defined by: MTI D

X

deg.v/ d.v/ C

v2V .G/

X

deg.v/2 ;

(5.22)

v2V .G/

where deg.v/ is the vertex degree. This index has found interesting applications in chemistry [NTM95]. Its discriminating power was investigated by Dobrynin [Do98] for cata-condensed benzenoid graphs. The average distance sum connectivity was introduced by Balaban [Ba82] and defined as follows: X m 1 J D .d.u/ d.v// 2 ; (5.23) mnC2 u;v2V .G/

where m is the number of edges in a graph G. This index is used successfully in drug design, and it is one of the widely used topological indices for quantitative structure– activity relationship and quantitative structure–property relationship [Ba83, Ba00, BBC94, DB99, ZT08]. Recently it was calculated for nanotubes, nanotori, and nanostars [YAK08]. The Randić index [Ra75] is based on the molecular connectivity and is achieved by the following formula: D

X u;v2V .G/

1

.deg.u/ deg.v// 2 :

(5.24)

114

E. Konstantinova

Table 5.1 The discriminating powers of indices on the polycyclic graphs N ID HDi I H Hexagonal graphs 849 285 0.999 0.999 0.997 0.993 Square graphs 298 382 0.997 0.995 0.954 0.811 Triangular graphs 295 365 0.984 0.982 0.844 0.466 Hexagonal graphs Square graphs Triangular graphs

W

HDn 0.999 0.998 0.992

H2 0.999 0.994 0.981

N

Hn

ID

J

M TI

W

849 285 298 382 295 365

0.997 0.906 0.585

0.659 0.133 0.021

0.998 0.993 0.986

0.0001 0.005 0.407

0.004 0.002 0.0008

0.0006 0.0003 0.0001

The numerical results of discriminating tests for the indices under consideration are given below on the sets of polycyclic graphs and trees.

4.1 Polycyclic Graphs The polycyclic graphs embedded to the regular hexagonal, square, and trigonal lattices up to 12, 13, and 16 cells correspondingly without holes are tested. The hexagonal graphs correspond to the structural formulae of planar polycyclic aromatic hydrocarbons [GC89,GC90]. The values of 12 information and topological indices were calculated for 849,285 hexagonal, 298,382 square, and 295,365 triangular graphs. The calculation accuracy for all indices is 1013 . The discriminating powers of indices were obtained in accordance with (5.21), and the final results are given in Table 5.1 where N is the number of graphs in the respective class (for more detailed results, see [KV03]). The data show that the information indices give much more discriminating powers. The indices HDn ; ID ; HDi ; H2 have the best result (S D 0:999) for hexagonal graphs. All topological indices, exception of J , could not discriminate between these graphs. The same situation is observed for square and triangular graphs. The degeneracy is high for W; M T I; and very low for W HDn ; ID . The information index I D discriminates not bad among hexagonal graphs, but it does not discriminate among square and triangular graphs. The opposite situation is observed for the Randić index . Its discriminating power is the lowest one on hexagonal graphs and the highest one on triangular graphs.

4.2 Trees Similar results were obtained on the set of trees. A tree is a connected acyclic graph. The discriminating powers of indices were calculated on the set of 3,490,528 trees up to 21 vertices. The data obtained are given in Table 5.2. The calculation accuracy for all indices is 1013 . The highest sensitivity corresponds to the information


115

Table 5.2 The discriminating powers of indices on 3,490,528 trees up to 21 vertices W

ID HDi I H HDn H2 Hn ID J M TI W 0.998 0.912 0.985 0.321 0.999 0.907 0.428 0.683 0.907 0.017 0.00004 0.00002

index HDn . There are no degeneracies of this index, i.e., S D 1, on the set of trees up to 17 vertices (the number of the trees is N D 81;134). There are two trees with p D 18 vertices; two trees with p D 19 vertices; eighth pairs of trees with p D 20 vertices (the number of these trees is N D 823;065), and six pairs of trees with p D 21 vertices (the number of these trees is N D 2;144;505) having the same values of information index HDn . The topological indices give a very low discriminating power on the trees. This investigation shows that the information index HDn can be used for characterizing molecular structures of alkane isomers. Thus, the data indicate that in common the information indices have greater discriminating powers than the topological ones. This confirms the main assumption that the information approach is much more appropriate in isomer discrimination.

5 Correlating Ability of Information Indices One of the key problems of modern chemistry is to find a relation between a structure and a reactive capability of a molecule. The reactive capability of a molecule can be characterized by its mass-spectrum, which contains the information on the ways of a molecule fragmentation and displays the “behavior” of some molecular fragments which can be interpreted as subgraphs of a molecular graph. Let us define the information index of the chemical mass-spectrum using the Shannon relation. From the information theory point of view, the mass-spectrum is the distribution of probabilities pi D AAi ; i D 1; : : : ; k; of the ions formation, P where Ai is the mass-spectrum amplitude of the i -th ion, A D kiD1 Ai , and k is the number of peaks in the mass-spectrum. The amplitude information index HA is defined as follows: HA D

k X Ai Ai log2 : A A i D1

(5.25)

On the other hand, a molecular graph that represents a structural formula of a molecule could be used for defining specific structural features of a molecule by means of information indices based on the distance in a molecular graph. As it was mentioned in Introduction section, the topological index is designed by transforming a molecular graph into a number, and it expresses in a numerical form the topology of the chemical species it presents. Moreover, it was shown by Skorobogatov et al. [SK91] that some information indices have a “chemical” spectral interpretation. Let us consider the information index H2 based on the

116

E. Konstantinova

Fig. 5.4 The distance vertex spectrum of a tree

ki

4 2 3

1

7 6

3 8

5

2

3

1 0

2

4

nλi 1

6

8

4 2 3

4, 5 7, 8 6

2

1

10 12 14 16 18 20

d(i)

7 6

3 2 1 0

4,5

7,8

1

1

6

2

2

3

4

8

5 1

5

6

7

8

9

10 autometricity classes

Fig. 5.5 The autometricity vertex spectrum of a tree

vertex distance d.i / and the number ki of vertices having the distance d.i /, and let us define the pairs .d.i /; ki / as the points in Euclidean plane. Then the distance vertex spectrum can be pictured on the plane by the lines f.d.i /; ki /; .d.i /; 0/g; i D 1; : : : ; k. Figure 5.4 shows the distance vertex spectrum for a tree. One more topological spectrum called the autometricity vertex spectrum was defined on the basis of a layer matrix. It could happen that some rows of this matrix are the same. This means that the corresponding vertices belong to one and the same class of autometricity. By this way, the vertex set is divided into the autometricity classes. The autometricity vertex spectrum is defined by the autometricity classes and the number ni of vertices in the i th class of autometricity. The rows of the matrix are ordered by their lengths, and then the rows are ordered lexicographically. The numeration of autometricity classes corresponds to the numeration of rows in the canonical layer matrix. Let us note that there is a finite probability scheme on the vertex set in accordance with the autometricity ratio, and one can define the autometricity information index Ha as follows: Ha D

k X n i

i D1

n

log2

ni : n

Figure 5.5 shows the autometricity vertex spectrum for the same tree.

(5.26)


117

HA 6 3

2

5

• ••

4 • 3

• ∗∗

1.25

•

•• ∗ ∗

• ∗∗

• •

• • ••

∗

•

1 ∗

∗

∗

CpFeC5H4R

2.25

3.25

4.25

5.25

Ha

1 : HA = 1.70 + 0.57 Ha (r = 0.940, n = 10) 2 : HA = 1.24 + 0.98 Ha (r = 0.975, n = 16) 3 : HA = 3.91 + 0.41 Ha (r = 0.940, n = 6)

Fig. 5.6 The correlations between the information indices Ha and HA on the set of ferrocene derivatives

The canonical layer matrix for this tree looks as follows: ˇ ˇ ˇ 1 2 3 1 ˇ 7; 8 ˇ ˇ ˇ 1 1 3 2ˇ 1 ˇ ˇ ˇ ˇ ˇ 3 3 1 ˇ 6 : Dˇ ˇ ˇ 2 3 2 ˇ 2 ˇ ˇ ˇ 1 3 3 ˇ 4; 5 ˇ ˇ ˇ 4 3 ˇ 3 The information indices H2 ; Ha and HA based on the topological as well as the chemical spectra provide the representation of a molecular graph and a chemical structure in terms of the same quantitative scale because their values are expressed in information bits. These indices are suitable ones for finding structure–activity correlations. At first, this approach was applied to the class of ferrocene derivatives Cp FeC5 H4 R, where R is a substituent [SK91]. The linear correlations between the information indices H2 and HA, and Ha and HA were found. It was shown that the initial set of molecular structures is divided into three subsets by the linear regression. In the cases considered, the correlation ratio ranges from 0.94 to 0.975. Figure 5.6 shows the correlations between the information indices Ha and HA on the set of ferrocene derivatives. The correlation ratio r and the number of structures n for each subset are presented.

118

E. Konstantinova

a

H

H

C

H

H H H

b

14

13

3 2

H Fe

H

12

H

H H

the structural formula of methylferrocene

c

1 1 4 4 λ= 1 10 4 4

−→

3 3 9 3 3 10 12 9

9 3 7 9 9 3 7 10

4 1

5 6

21 11

H

7 3 9 7 3 7 10

20

22 23 24

15

10

16 7 17

8 9

18

19 the molecular graph of methylferrocene 12, 13, 17, 18, 19, 20, 21 = X1 22, 23, 24, = X2 = X3 1, 2, 7, 8, 9, 10, 11 15 = X4 14, 16 = X5 6 = X6 4 = X7 = X8 3, 5

−→

the canonical layer matrix of the molecular graph

d nλi 7 6 5 −→ 4 3 2 1 0

1

2 3 4 5 6 7 8 9 autometricity classes Xi the autometricity vertex spectrum

Fig. 5.7 The example of the autometricity vertex spectrum for the molecular graph of methylferrocene

The similar results were obtained for information indices H2 and HA. The correlations between them were found as follows: 1 W HA D 1:09 C 0:76 H2 .r D 0:950; n D 9/; 2 W HA D 1:30 C 0:99 H2 .r D 0:975; n D 17/; 3 W HA D 3:96 C 0:40 H2 .r D 0:950; n D 6/: The example of the autometricity vertex spectrum for a molecular structure of this class is given in Fig. 5.7.


119

HA 1

5

2 3

∗ 4

∗

• • •

∗

3 ∗ ∗

∗ ∗

∗ ∗ ∗

∗ ∗

• • • • •

PhMeSiR

2

V inMeSiR

• •

PhAll(V in)SiR

1 1.25

2.25 3.25 4.25 5.25 1 : HA = −5.9 + 2.9 H2 (r = 0.945, n =12) 2 : HA = −8.3 + 2.8 H2 (r = 0.982, n =10) 3 : HA = −8.5 + 3.4 H2 (r = 0.856, n =5)

H2

Fig. 5.8 The correlations between the information indices H2 and HA on the set of arylsilanes

Later, this approach was used for finding spectrum–structure correlations and investigating relationships between the structure and reactivity of molecules in the gas phase on several classes (20) of organic and organometallic compounds by Prof. Yu. S. Nekrasov in cooperation with a team at Nesmeyanov Institute of Organoelement Compounds [NST93, NS93, NS96, NS02, NST05, SN93]. It has been shown with the example of some classes of organoelement compounds that the information indices of the topological spectra of molecular graphs correlate with the structural or electronic parameters of the compounds being studied. In particular, such correlations were found for cymantrene derivatives [NS02] and also on the set of 28 arylsilanes [NST93]. In the last case, it was shown that there are three subsets of arylsilanes and each of them has specific structure peculiarities. Figure 5.8 shows the correlations between the information indices H2 and HA on the set of arylsilanes. Line 1 corresponds to the set of phenylmethylsilanes, line 2 corresponds to the set of phenylallyl- and phenylvinylsilanes, and line 3 corresponds to the set of vinylmethylsilanes. As one can see from the picture, the correlation ratio for all cases ranges from 0.856 to 0.982. The results thus obtained can be used in designing compact database retrieval systems for the structures and mass-spectra of organosilicon compounds.

120

E. Konstantinova

It is known that in determining spectrum–structure correlations, the choice of parameters characterizing the mass-spectrum and structure of a compound is a critical problem. According to the results obtained at Nesmeyanov Institute of Organoelement Compounds of Russian Academy of Sciences, one can say that the information-theoretic method proposed by Skorobogatov et al. [SK91] based on the topological spectra of molecular graphs and on the mass-spectra of chemical compounds may be used as the general approach in finding spectrum–structure correlations and investigating relationships between the structure and reactivity of molecules in the gas phase.

6 Conclusion Information theory is a useful tool in developing the new invariants for characterizing molecular structures and investigating structure–property and structure–activity relationships. The only limit to the design of invariants is the imagination and resourcefulness of investigators. The situation is similar to a search for a system of codes and a design of codes for chemical structures in particular. There are graph invariants considered in the mathematical literature, which have not yet been tested for possible chemical applications. Such may or may not be of interest in chemistry, but without testing we will not know. If some existing mathematical invariants are shown to correlate with some of the known molecular properties, the findings may be of considerable interest for chemistry – not only because such invariants may offer a novel predictive tool in structure–property or structure–activity studies, or an alternative route to a certain molecular property, but they may give additional insight into these relationships. Moreover, as it was shown the information-theoretic approach is effectively used in spectrum–structure correlations and investigating relationships between the structure and reactivity of molecules of organoelement compounds in the gas phase. Such results may show novel mathematical concepts of use in chemistry, and not only in chemistry. Recently, the information-theoretic approach was applied to general complex networks which results in non-partition-based entropy measures for determining their information content [DE08]. Moreover, this method was applied to hierarchical molecular networks where information inequalities for such graphs were also inferred [DBE08]. This new method is based on local vertex functionals such that one can assign a probability value to each individual vertex in a graph without finding an equivalence relation. Several parametric information functionals based on metrical properties of graphs have been considered there. It would be very interesting to apply the information-theoretic approach for investigating some other networks, for example, interconnection networks in computer science which are widely represented by graphs, in particular, Cayley graphs such as the Hamming graph also well known in coding theory, the star graph, the pancake graph, the butterfly graph, and so on. All these graphs are regular and vertex-transitive, and some of them are distance-transitive graphs which means that it will be not so


121

easy to find information indices characterizing these graphs. However, applying the information-theoretic approach gave us the good results in chemistry and may give a new view on the problems in other areas. Acknowledgment The research was supported by the RFBR grant 09–01–00244.

References [As56] [AC72]

[AC74]

[Ba76] [Ba82] [Ba83] [Ba85] [BX71] [BC80] [BBC94]

[Ba00]

[BF69] [B99]

[BRG80]

[BM83]

[BNV91]

Ashby, W.R.: An Introduction to Cybernetics. Wiley, New York (1956) Aslangul, C., Constanciel, R., Daudel, R., Kottis, P.: Aspects of the localizability of electrons in atoms and molecules: Loge theory and related methods. Adv. Quant. Chem. 6, 93–141 (1972) Aslangul, C., Constanciel, R., Daudel, R., Esnault, L., Ludena, E.: The Loge theory as a starting point for variational calculations. I. General formalism. Int. J. Quant. Chem. 8, 499–522 (1974) Balaban, A.T., (ed.): Chemical Applications of Graph Theory. Academic, New York (1976) Balaban, A.T.: Highly discriminating distance-based topological indices. Chem. Phys. Lett. 89, 399–404 (1982) Balaban, A.T.: Topological indices based on topological distances in molecular graphs. Pure Appl. Chem. 55, 199–206 (1983) Balaban, A.T.: Applications of graph theory in chemistry. J. Chem. Inf. Comput. Sci. 25, 334–343 (1985) Balaban, A.T., Harary, F.: The characteristic polynomial does not uniquely determine the topology of a molecule. J. Chem. Doc. 11, 258–259 (1971) Balaban, A.T., Chirac, A., Motoc, I., Simon, Z.: Steric Fit in Quantitative Structure– Activity Relationships. Lecture Notes in Chemistry. Springer, Berlin (1980) Balaban, A.T., Basak, S.C., Colburn, T., Grunwald, G.D.: Correlation between structure and normal boiling points of Haloalkanes C1–C4 using neural networks. J. Chem. Inf. Comp. Sci. 34, 1118–1121 (1994) Balaban, A.T.: QSAR and computational methods in drug discovery, In: Meyers, R.A. (ed.) Encyclopedia of Analytical Chemistry, vol. 8, pp. 7288–7311. Wiley, Chichester (2000) Ban, T., Fujita, T.: Mathematical approach to structure-activity study of sympathomimetic amines. J. Med. Chem. 12, 353–356 (1969) Basak, S.C.: Information theoretic indices of neighborhood complexity and their applications, In: Devillers, J., Balaban, A.T. (eds.) Topological Indices and Related Descriptors in QSAR and QSPR, pp. 563–593. Gordon and Breach Science Publishers, The Netherlands (1999) Basak, S.C., Roy, A.B., Ghosh, J.J.: Study of the structure–function relationship of pharmacological and toxicological agents using information theory, In: Avula, X.J.R., Bellman, R., Luke, Y.L., Rigler, A.K. (eds.) Proceeding of the 2nd International Conference on Mathematical Modelling, vol.2, pp. 851–856. University of Missouri– Rolla, Rolla, Missouri (1980) Basak, S.C., Magnusson, V.R.: Molecular topology and narcosis: a quantitative structure-activity relationship (QSAR) study of alcohols using complementary informartion content (CIC). Arzneim. Forsch. Drug Res. 33, 501–503 (1983) Basak, S.C., Niemi, G.J., Veith, G.D.: Predicting properties of molecules using graph invariants. J. Math. Chem. 7, 243–272 (1991)

122 [BBG93]

[BG93] [BG95] [BBG00] [BGB04] [BL72]

[BLB72]

[Bon79] [Bon81] [Bon83] [BKK76] [BT77] [BT78] [BKT79] [BMT80] [BMT81] [BT82] [Bril56] [CS78] [CBF68] [DB74] [DBE08] [DE08] [D96] [DB99] [DG98]

E. Konstantinova Basak, S.C., Bertelsen, S., Grunwald, G.D.: Application of graph theoretical parameters in quantifying molecular similarity and structure–activity studies. J. Cem. Inf. Comput. Sci. 34, 270–276 (1993) Basak, S.C., Grunwald, G.D.: Use of graph invariants, volume and total surface area in predicting boiling point of alkanes. Math. Model. Sci. Comp. 2, 735–740 (1993) Basak, S.C., Grunwald, G.D.: Molecular similarity and estimation of molecular properties. J. Chem. Inf. Comput. Sci. 35, 366–372 (1995) Basak, S.C., Balaban, A.T., Grunwald, G.D., Gute, B.D.: Topological indices: their nature and mutual relatedness. J. Chem. Inf. Comput. Sci. 40, 891–898 (2000) Basak, S.C., Gute, B.D., Balaban, A.T. Interrelationship of major topological indices evidenced by clustering. Croat. Chem. Acta CCACAA 77, 331–344 (2004) Bernstein, R.B., Levine, R.D.: Entropy and chem. change. I. Characterization of product (and reactant) energy distributions in reactive molecular collisions: information and entropy deficiency. J. Chem. Phys. 57, 434–449 (1972) Ben-Shaul, A., Levine, R.D., Bernstein, R.B.: Entropy and chem. change. II. Analysis of product energy distributions: temperature and entropy deficiency. J. Chem. Phys. 57, 5427–5447 (1972) Bonchev, D.: Information indices for atoms and molecules. MATCH Commun. Math. Comput. Chem. 7, 65–113 (1979) Bonchev, D.: Information theory interpretation of the Pauli principle and Hund rule. Int. J. Quant. Chem. 19, 673–679 (1981) Bonchev, D.: Information–theoretic Indices for Characterization of Chemical Structures. Research Studies Press, Chichester (1983) Bonchev, D., Kamenski, D., Kamenska, V.: Symmetry and information content of chemical structures. Bull. Math. Biophys. 38, 119–133 (1976) Bonchev, D., Trinajstić, N.: Information theory, distance matrix, and molecular branching. J. Chem. Phys. 38, 4517–4533 (1977) Bonchev, D., Trinajstić, N.: On topological characterization of molecular branching. Int. J. Quant. Chem. S12, 293–303 (1978) Bonchev, D., Knop, J.V., Trinajstić, N.: Mathematical models of branching. MATCH Commun. Math. Comput. Chem. 6, 21–47 (1979) Bonchev, D., Mekenyan, O., Trinajstić, N.: Topological characterization of cyclic structure. Int. J. Quant. Chem. 17, 845–893 (1980) Bonchev, D., Mekenyan, O., Trinajstić, N.: Isomer discrimination by topological information approach. J. Comput. Chem. 2, 127–148 (1981) Bonchev, D., Trinajstić, N.: Chemical information theory. Structural Aspects. Int. J. Quant. Chem. Symp. 16, 463–480 (1982) Brillouin, L.: Science and Information Theory. Academic, New Nork (1956) Chapman, N.B., Shorter J. (eds.): Correlation Analysis in Chemistry. Plenum, New York (1978) Crum-Brown, A., Fraser, T.R.: Trans Royal Soc. Edinburgh 25, 151–203, 257–274, 693–739 (1868-1869) Daudel, R., Bader, R.F., Stephens, M.E., Borett, D.S.: The electron pair in chemistry. Can. J. Chem. 52, 1310–1320 (1974) Dehmer, M., Borgert, S., Emmert–Streib, F.: Entropy bounds for hierarchical molecular networks. PLoS ONE 3, e3079 (2008) Dehmer, M., Emmert–Streib, F.: Structural information content of networks: graph entropy based on local vertex functionals. Comput. Biol. Chem. 32, 131–138 (2008) Devillers, J. (ed.): Genetic Algorithms in Molecular Modeling (Principles of QSAR and Drug Design). Academic, London (1996) Devillers, J., Balaban, A.T. (eds.).: Topological Indices and Related Descriptors in QSAR and QSPR. Gordon and Breach, Amsterdam, Netherlands (1999) Diudea, M.V., Gutman, I.: Wiener–type topological indices. Croat. Chem. Acta CCACAA 71, 21–51 (1998)

5 Information-Theoretic Methods in Chemical Graph Theory [Do98] [DG77] [DLA73] [DLA74]

[En76] [En97] [F94] [FBE80]

[FEP82]

[FW64] [FB71]

[Gu94] [GC89] [GC90] [HSK72] [H77] [H80] [H81] [HL95] [Ha69] [HGL90]

[Ho71]

[JBM88] [JM90] [K00]

123

Dobrynin, A.A.: Discriminating power of the Schultz index for cata–condensed benzenoid graphs. MATCH Commun. Math. Comput. Chem. 38, 19–32 (1998) Doyle, J.K., Graver, J.E.: Mean distance in graph. Discrete Math. 17, 147–154 (1977) Dubois,J.E., Laurent, D., Aranda, A.: Perturbation of environments which are limited, concentric and ordered. J. Claim. Phys. 11, 1608–1616 (1973) Dubois, J.E.: DARC system in chemistry, In: Wipke, W.T., Heller, S., Fellmann, R., Hyde, E. (eds.) Computer Representation and Manipulation of Chemical Information, pp. 239–263. Wiley, New York (1974) Entringer, R.C., Jackson, D.E., Snyder, D.A.: Distance in graphs. Czechoslovak Math. J. 2, 283–297 (1976) Entringer, R.C.: Distance in graphs: trees. J. Combin. Math. Combin. Comput. 24, 65–84 (1997) Fisher, E.: Einfluss der Configuration auf die Wirkung der Enzyme. Chem. Ber. 74, 70–77 (1986) Fratev, F., Bonchev, D., Enchev, V.: A theoretical information approach to ring and total aromaticity in ground and excited states. Croat. Chem. Acta CCACAA 53, 545–554 (1980) Fratev, F., Enchev, V., Polansky, O.E., Bonchev, D.: A theoretical-information study on the electron delocalization (aromaticity) of annulenes with and without bond alternation. THEOCHEM 88, 105–118 (1982) Free, S.M., Wilson, I.W.: A mathematical contribution to structure–activity studies. J. Med. Chem. 7, 395–399 (1964) Fujita, T., Ban, T.: Structure-activity relation 3. Structure-activity study of phenethylamines as substrates of biosynthetic enzymes of sympathetic transmitters. J. Med. Chem. 14, 148–152 (1971) Gutman, I.: Selected properties of the Schultz molecular index. J. Chem. Inf. Comput. Sci. 34, 1087–1089 (1994) Gutman, I., Cyvin, S.J.: Introduction to the Theory of Benzenoid Hydrocarbons. Springer, Berlin (1989) Gutman, I., Cyvin, S.J. (eds): Advances in the Theory of Benzenoid Hydrocarbons. Springer, Berlin (1990) Hansch, C., Schaeffer, J., Kerley, R. Alcohol dehydrogenase structure–activity relationships. J. Biol. Chem. 247, 4703–4710 (1972) Hansch, C.: On the predictive value of QSAR. In: Buisman, K. (ed.) Biological Activity and Chemical Structure, pp.47–61. Elsevier, Amsterdam (1977) Hansch, C.: Use of quantitative structure-activity relationships (QSAR) in drug design (review). Pharm. Chem. J. 14, 678–691 (1980) Hansch, C.: The physicochemical approach to drug design and discovery (QSAR). Drug Dev. Res. 1, 267–309 (1981) Hansch C., Leo A.: Exploring QSAR Fundamentals and Applications in Chemistry and Biology. ACS, Washington DC, USA (1995) Harary, F.: Graph Theory. Addison–Wesley, Reading, MA (1969) Hernandez-Gallegos, Z., Lehmann, F.P.A. : A Free–Wilson/Fujita–Ban analysis and prediction of the analgesic potency of some 3-hydroxy- and 3-methoxy-Nalkylmorphinan-6-one opioids. J. Med. Chem. 33, 2813–2817 (1990) Hosoya, H.: Topological index. A newly proposed quantity characterizing the topological nature of structural isomers of hydrocarbons. Bull. Chem. Soc. Jpn. 44, 2332–2339 (1971) Johnson, M., Basak, S.C., Maggiora, G.: A characterization of molecular similarity methods for property prediction. Math. Comput. Modell. 11, 630–6634 (1988) Johnson, M., Maggiora, G.: Concepts and Applications of Molecular Similarity. Wiley, New York (1990) Karelson, M.: Molecular descriptors in QSAR/QSPR. Wiley, New York (2000)

124 [KL94]

E. Konstantinova

Karlitzky, A.R., Lobanov, V.S.: CODESSA, Version 5.3, University of Florida, Gainesville (1994) [KH76] Kier, L.B., Hall, L.H.: Molecular Connectivity in Chemistry and Drug Research. Academic, New York (1976) [KH81] Kier, L.B., Hall, L.H.: Derivation and significance of valence molecular connectivity. J. Pharm. Sci. 70, 583–589 (1981) [KH86] Kier, L.B., Hall, L.H.: Molecular Connectivity in Structure–Activity Analysis. Research Studies Press, Letchworth (1986) [KR87] King, R.B., Rouvray, D.H. (eds.): Graph Theory and Topology in Chemistry. Elsevier, Amsterdam (1987) [Kol69] Kolmogorov, A.N.: On logic basis of information theory. Probl. Peredachi Inf. 5, 3–7 (1969) [KP90] Konstantinova, E.V., Paleev, A.A.: Sensitivity of topological indices of polycyclic graphs. Vychisl. Sistemy 136, 38–48 (1990) [Kon96] Konstantinova, E.V.: The discrimination ability of some topological and information distance indices for graphs of unbranched hexagonal systems. J. Chem. Inf. Comput. Sci. 36, 54–57 (1996) [KD00] Konstantinova, E.V., Diudea, M.V.: The Wiener polynomial derivatives and other topological indices in chemical research. Croat. Chem. Acta CCACAA 73, 383–403 (2000) [KV03] Konstantinova, E.V., Vidyuk, M.V.: Discriminating tests of information and topological indices. Animals and trees. J. Chem. Inf. Comp. Sci. 43, 1860–1871 (2003) [K88] Kubinyi, H.: Free–Wilson Analysis. Theory, applications and its relationship to Hansch analysis. Quant. Struct. Act. Relat. 7, 121–133 (1988) [K93] Kubinyi, H.: QSAR: Hansch Analysis and Related Approaches. In: Mannhold, R., Kroogsgard-Larsen, P., Timmerman, H. (eds.) Methods and Principles in Medicinal Chemstry. VCH, Weinheim (1993) [L90] Lajiness, M.S.: Molecular similarity–based methods for selecting compounds for screening. In: Rouvray, D.H. (ed.) Computational Chemical Graph Theory. Nova Science Publishers, New York (1990) [MH83] Magnuson, V.R., Harris, D.K., Basak, S.C.: Topological indices based on neighborhood symmetry: chemical and biological application. In: King, R.B. (ed.) Chemical Applications of Topology and Graph Theory, pp. 178–191. Elsevier, Amsterdam (1983) [Mar71] Marshall, C.W.: Applied Graph Theory. Wiley-Interscience, New York (1971) [MB86] Mekenyan, O., Bonchev, D.: OASIS method for predicting biological activity of chemical copounds. Acta Pharm. Jugosl. 36, 225–237 (1986) [MMB93] Mekenyan, O., Mercier, C., Bonchev, D., Dubois, J.E.: Comparative study of DARC/PELCO and OASIS methods. II. Modelling PNMT inhibitory potency of benzylamines and amphetamines. Eur. J. Med. Chem. 28, 811–819 (1993) [MC03] MolConnZ, Ver.4.05, Hall Ass. Consult., Quincy, MA (2003) [Mo65] Morgan, H.L.: The generation of a unique machine description of chemical structures – a technique developed at chemical abstracts service. J. Chem. Doc. 5, 107–113 (1965) [Mor55] Morovitz, H.: Some order–disorder considerations in living systems. Bull. Math. Biophys. 17, 81–86 (1955) [Mo68a] Mowshovitz, A.: The information content of digraphs and infinite graphs. Bull. Math. Biophys. 30, 225–240 (1968) [Mo68b] Mowshovitz, A.: An index of the relative complexity of a graph. Bull. Math. Biophys. 30, 175–204 (1968) [Mo68c] Mowshovitz, A.: Graphs with prescribed information content. Bull. Math. Biophys. 30, 387–414 (1968) [Mo68d] Mowshovitz, A.: Entropy measures and graphical structure. Bull. Math. Biophys. 30, 533–546 (1968)

5 Information-Theoretic Methods in Chemical Graph Theory [NST93] [NS93]

[NS96]

[NS02]

[NST05]

[NTM95] [P52] [PBC73] [QS81] [Ra75] [Ra79]

[Ra84] [Ra91] [RT88] [Rash55] [Rash60] [RR84]

[RCD85] [RBT88]

[Rou83]

[Rou89] [RB79]

125

Nekrasov, Yu.S., Tepfer, E.E., Sukharev, Yu.N.: On the relationship between the mass– spectral and structural indices of arylsilanes. Russ. Chem. Bull. 42, 343–346 (1993) Nekrasov, Yu.S., Sukharev, Yu.N., Molgacheva, N.S., Tepfer, E.E.: Generalized characteristics of mass–spectra of aromatic compounds and their correlation with the constants of substituents. Russ. Chem. Bull. 42, 1986–1990 (1993) Nekrasov, Yu.S., Sukharev, Yu.N., Tepfer, E.E., Molgacheva, N.S.: Establishment of correlations between the structure and reactivity of molecules in the gas phase based on information theory. Russ. Chem. Bull. 45, 2542–2546 (1996) Nekrasov, Yu.S., Sukharev, Yu.N., Tepfer, E.E., Yakushin, S.: Electron impact mass spectra data processing for evaluation of gas–phase reactivity of cymantrene (tricarbonyl 5–cyclopentadienylmanganese) derivatives. Eur. J. Mass Spectrom. 8, 247–251 (2002) Nekrasov, Yu.S., Sukharev, Yu.N., Tepfer, E.E.: Determination of spectrum–structure correlations based on integral parameters of mass–spectra. J. Analyt. Chem. 20, 1035– 1037 (2005) Nikolić, S., Trinajstić, N., Mihalić, Z.: The Wiener index: developments and applications. Croat. Chem. Acta CCACAA 68, 105–129 (1995) Platt, J.R.: Prediction of isomeric differences in paraffin properties. J. Phys. Chem. 56, 328–336 (1952) Purcell, W.P., Bass, G.E., Clayton, J.M.: Strategy in Drug Design. A Molecular Guide to Biological Activity. Wiley–lnterscience, New York (1973) Quintas, L.V., Slater, P.J.: Pairs of non–isomorphic graphs having the same path degree sequence. MATCH Commun. Math. Comput. Chem. 12, 75–86 (1981) Randić, M.: On characterization of molecular branching. J. Am. Chem. Soc. 69, 6609– 6615 (1975) Randić, M.: Characterization of atoms, molecules and classes of molecules based on paths enumerations. Proc. Bremen Konferenz zur Chemie Univ. Bremen. 2, 5–64 (1979) Randić, M.: On molecular identification numbers. J. Chem. Inf. Comput. Sci. 24, 164–175 (1984) Randić, M.: Generalized molecular descriptors. J. Math. Chem. 7, 155–168 (1991) Randić, M., Trinajstić, N.: Composition as a method for data redustion: Application to carbon-13 NMR chemical shifts. Theor. Chim. Acta 73, 233–246 (1988) Rashevsky, N.: Life, information theory and topology. Bull. Math. Biophys. 17, 229–235 (1955) Rashevsky, N.: Life, information theory, probability and physics. Bull. Math. Biophys. 22, 351–364 (1960) Raychaudhary, C., Ray, S.K., Ghosh, J.J., Roy, A.B., Basak, S.C.: Discrimination of isomeric structures using information theoretic topological indices. J. Comput. Chem. 5, 581–588 (1984) Razinger, M., Chretien, J.R., Dubois, J.K.: Structural selectivity of topological indices in alkane series. J. Chem. Inf. Comput. Sci. 25, 23–27 (1985) Romero, D.L., Busso, M., Tan, C.K., Reusser, F., Palmer, J.R., Poppe, S.M., Aristoff, P.A., Downey, K.M., So, A.G., Resnick, L., Tarpley, W.G.: Nonnucleoside reverse transcriptase inhibitors that potently and specifically block human immunodeficiency virus type I replication. Proc. Natl. Acad. Sci. USA. 88, 8806–8810 (1991) Rouvray, D.H.: Should we have designs on topological indices? In: Chemical applications of topology and graph theory. In: King, R.B. (ed.) Studies in Physical and Theoretical Chemistry. Elsevier, Amsterdam (1983) Rouvray, D.H.: The limits of applicability of topological indices. J. Mol. Struc. (Theochem) 185, 187–201 (1989) Rouvray, D.H., Balaban, A.T.: Chemical applications of graph theory. In: Wilson, R.J., Beineke, L.W. (eds.) Applications of Graph Theory, pp. 177–221. Academic, New York (1979)

126 [RB91] [SHP81]

[S03]

[Sh49] [SK91]

[Sl82] [Spi63]

[Spi64a] [Spi64b] [SBJ79] [SN93]

[Syl78]

[TC00] [TCM06]

[Tri92] [Tr56a] [Tr56b] [Val63] [Wi47] [Wi48a] [Wi48b] [YAK08] [ZT08]

E. Konstantinova Rouvray, D.H., Bonchev, D.: Chemical Graph Theory: Introduction and Fundamentals. Abacus Press, Tunbridge Wells, Kent (1991) Schaad, L.J., Hess (Jr.) B.A., Purcell, W.P., Cammarata, A., Franke, R., Kubinyi, H.: Compatibility of the Free–Wilson and Hansch quantitative structure–activity relations. J. Med. Chem. 24(7), 900–901 (1981) Selassie, C.D.: History of Quantitative Structure–Activity Relationships, In: Abraham, D.J. (ed.) Burger’s Medicinal Chemistry and Drug Discovery, pp. 1–48. Wiley, New York (2003) Shannon, C., Weaver, W.: Mathematical Theory of Communications. University of Illinois, Urbana (1949) Skorobogatov, V.A., Konstantinova, E.V., Nekrasov, Yu.S., Sukharev, Yu.N., Tepfer, E.E.: On the correlation between the molecular information topological and mass– spectra indices of organometallic compounds. MATCH Commun. Math. Comput. Chem. 26, 215–228 (1991) Slater, P.J.: Counterexamples to Randić’s conjecture on distance degree sequences for trees. J. Graph Theory 6, 89–92 (1982) Spialter, L.: The atom connectivity matrix (ACM) and its charactereistic polynimial (ACMCP): a new computer–oriented chemical nomenclature. J. Am. Chem. Soc. 85, 2012–2013 (1963) Spialter, L.: The atom connectivity matrix (ACM) and its charactereistic polynimial (ACMCP). J. Chem. Doc. 4, 261–269 (1964) Spialter, L.: The atom connectivity matrix characteristic polynimial (ACMCP) and its physico–geometric (topological) significance. J. Chem. Doc. 4, 269–274 (1964) Stuper, A., Brugger, W., Jurs, P.: Computer Assisted Studies of Chemical Structure and Biological Function. Wiley, New York (1979) Sukharev, Yu.N., Nekrasov, Yu.S., Molgacheva, N.S., Tepfer, E.E.: Computer processing and interpretation of mass–spectral information. Part IX - Generalized characteristics of mass–spectra. Org. Mass Spectrom. 28, 1555–1561 (1993) Sylvester, J.J.: On an application of the new atomic theory to the graphical representation of the invariants and covariants of binary quantics, with three appendices. Am. J. Math. 1, 64–125 (1878) Todeschini, R., Consonni, V.: Handbook of Moleculat Descriptors. Wiley, Weinheim, Germany (2000) Todeschini, R., Consonni, V., Mauri, A., Pavan, M.: DRAGON – Software for the calculation of molecular descriptors. Ver.5.4 for Windows, Talete srl, Milan, Italy (2006) Trinajstić, N.: Chemical Graph Theory, 2nd edn. (revised). CRC Press, Boca Raton, FL (1992) Trucco, E.: A note of the information content of graphs. Bull. Math. Biophys. 17, 129–135 (1956) Trucco, E.: On the informational content of graphs-compound symbols. Different states for each point. Bull. Math. Biophys. 18, 237–245 (1956) Valentinuzzi, M., Valentinuzzi, M.E.: Information content of chemical structures and some possible biological applications. Bull. Math. Biophys. 25, 11–27 (1963) Wiener, H.: Structural determination of paraffin boiling points. J. Am. Chem. Soc. 69, 17–20 (1947) Wiener, H.: Vapor pressure–temperature relationships among the branched paraffin hydrocarbons. J. Phys. Chem. 52, 425–430 (1948) Wiener, H.: Relation of the physical properties of the isomeric alkanes to molecular structure. J. Phys. Chem. 52, 1082–1089 (1948) Yousefi-Azari, H., Ashrafi, A.R., Khalifeh, M.H.: Topological indices of nanotubes, nanotori and nanostars. Dig. J. Nanomater. Bios. 3, 251–255 (2008) Zhou, Bo; Trinajstic, N.: Bounds on the Balaban index. Croatia Chemica Acta CCACCA 81, 319–323 (2008)

Chapter 6

On the Development and Application of Net-Sign Graph Theory Prabhat K. Sahu and Shyi-Long Lee

Abstract This report briefly describes the development and applications of net-sign graph theory. The current work enunciates the graph (molecule) signature of nonalternant non-benzenoid hydrocarbons with odd member of rings (non-bipartite molecular graphs) based on chemical signed graph theory. Experimental evidences and Hückel spectrum reveal that structure possessing nonbonding molecular orbital (NBMOs) is very unstable and highly reactive under the drastic conditions of low temperature. Chemical signed graph theoretical approach is applied successfully to classify the non-bipartite molecular graphs with a view to Randic’s conjugated circuit models based on their spectral characteristic. The obtained results based on net-sign approach are compared with those obtained using Hückel calculations. Keywords Net-sign graphs • Non-bipartite molecular graphs • Spectral characteristic MSC2000 Primary 05C22, 05C40; Secondary 65S05, 94C15.

1 Introduction Elucidation of the connection between molecular structure and mathematical graph has been main impetus for the graph theoreticians to introduce graph theory to the chemical community [1]. The topological information content in a chemical graph has been interesting and intriguing to theoretical chemists for more than a century. The basic ideas implemented for the characterizations of the bonding topology of molecular graph are based on two different levels: (a) external level: the components S.-L. Lee () Department of Chemistry and Biochemistry, National Chung Cheng University, Chia-Yi, 621, Taiwan e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 6, © Springer Science+Business Media, LLC 2011

127

128

P.K. Sahu and S.-L. Lee

are atomic connections, i.e., atoms and bonds in molecule that correspond to vertex and edge graph [2–18], and (b) internal level: the constituents are electronic structure, i.e., contribution of atomic orbitals and overlap interactions among atomic orbitals in a molecular orbital. The concept of internal connectivity was first introduced by Lee et al. [19] so as to study systematically net-sign analysis of eigenquantities of an MO graph. Internal connectivity is the microscopic topological property hidden behind a molecular graph; it spans the entire graph space; it records all information concerning the contributions of each vertex and the nature of the connections among any pair of vertices. The adjacency matrix is one of the basic topological matrices of a graph. The equivalence between adjacency matrices of graph theory and secular matrices of Hückel theory has been pointed out by several authors [20–23]. Based on this equivalence, the internal topological feature embedded in the eigenvectors of the adjacency matrix is extracted to build up signed graph. The graph of each eigenvector was found to belong to the class of vertexsigned graphs (VSGs) and the graphs characterizing the connections are edge-signed graphs (ESGs). VSGs and ESGs thus described the internal connectivity of the molecular graph. These were also found to be useful in describing bonding graph, nonbonding graphs, and antibonding graphs. The theoretical background for the net-sign analysis has been well explained in detail, and it also confirms to be a direct consequence of the oscillating theorem [24, 25]. The ambiguity raised for degeneracy has also been successfully solved for the modified version of the net-sign analysis [26].

1.1 Theoretical Background One of the general properties of motion in one dimension has been stated in the oscillating theorem [27] that the function f .x/ corresponding to the .n C 1/th eigenvalue vanishes n times, i.e., f .x/ has n nodes. This can be easily shown by the spectrum of a one-dimensional box and the spectrum of a one-dimensional harmonic oscillator [28]. Net-sign analysis of the spectra of a chemical graph [19] is a direct consequence of this oscillating theorem. Consider an N-vertex one-dimensional graph which has N 1 edges that can contribute the sign to the net sign of an ESG. The lowest level has no node which leads to a net sign of N 1. The second-lowest energy level has one node which leads to a net sign of N 3. The nth energy level has n 1 nodes which result in a net sign of N 2n 1. Therefore, the net-sign analysis is valid in one-dimensional graphs. For two-dimensional systems, the dependence of energy eigenvalues on the nodal property is not clear [29]. Using a two-dimensional harmonic oscillator and a twodimensional box as two topological limits, Lee and Lin [30] conjectured that the planar version of oscillating theorem could not exist. The same authors also pointed out that the net-sign analysis for simple planar graphs would yield similar results as those of a two-dimensional harmonic oscillator [30].

6 On the Development and Application of Net-Sign Graph Theory

129

In essence, for a two-dimensional harmonic oscillator, the energy can be expressed as: E˛ .Six C Siy /

(6.1)

and for a two-dimensional box the energy is: E˛.Six Nx 1/2 C .Siy Ny 1/2 ;

(6.2)

where Six and Siy are net signs over edges along the x and y directions, and Nx and Ny are number of vertices in the x and y directions, respectively.

1.2 Net-Sign Approach Let G be an n-vertex molecular graph and A(G) its adjacency matrix. The eigenvalues 1 ; 2 ; : : :; n and the eigenvectors 1 ; 2 ; : : :; n of G satisfy the equations: A.G/i D i i ; i D 1; 2; : : : ; n: Let further i D .i1 ; i 2 ; : : :; in /T . Then the net sign associated with the eigenvector i and the eigenvalue i is defined as: Si D

X

sign .ir ; is /;

(6.3)

r<s

where the summation goes over all pairs of adjacent vertices of the graph G and where sign(x) is equal to Cl; 0, and 1 for x > 0; x D 0, and x < 0, respectively. The net signs can be evaluated directly from (6.3). It has also been proposed [24, 25] a general method to generate the VSGs or topological orbitals without solving the respective eigenvectors of the adjacency matrix. If we consider a molecular graph G defined on N vertices and each vertex of G is then randomly given a sign of either plus or minus, we will get a total number of 2N configurations of VSGs. As it has been described [19] that the VSGs can be used to represent Hückel molecular orbitals, and in a molecular orbital only the relative sign, but not the absolute sign, is of topological significance. Thus, the total number of VSGs needed to be considered is simply 2N 1 configurations. Thereby, one can start from one vertex with minus sign and generate exactly the same set of VSGs as that produced from the same vertex starting with a positive sign. In each 2N 1 VSGs, the sign of an edge can be determined by the sign of (Cir Cis ), where Cir and Cis are coefficients of the ith molecular orbital on vertexes r and s. In this approach, now we can build the corresponding ESGs for each VSG. The net sign of each VSG

130


can then be determined approximately by the net sign of the corresponding ESG as defined by (6.3a), very similar to that of (6.3). Si D

X

sign.Cir ; Cis /;

(6.3a)

r<s

where the summation goes over all pairs of adjacent vertices of the graph G and where sign(q) is equal to Cl; 0; and 1 for q > 0; q D 0; and q < 0, respectively. 1.2.1 Model Graphs We consider two model graphs, G43 and b-G44 to illustrate the topological features of the internal connectivity of a graph. G43 The G43 graph is the simplest 4-vertex connected graph. Molecules having such a graph are n-butane and butadiene. For a complete description of the multibonds in the butadiene, both ¢ and connections are included in this category. In fact, the adjacency matrix of a graph has a block diagonal structure provided that the ¢– separation is valid. Each block along the diagonal corresponds to one kind of connection in the chemical graph. For G43 , the adjacency matrix, A (G43 ), is given by: 0

0 B1 A.G43 / D B @0 0

1 0 1 0

0 1 0 1

1 0 0C C: 1A 0

Eigenvalues and eigenvectors result from the diagonalization of A.G43 /. The eigenvalues, i .G43 /, and the corresponding eigenvectors, i .G43 /, are: 1 2 3 4

D 1:62; D 0:62; D 0:62; D 1:62;

1 2 3 4

D .˙0:37; ˙0:60; ˙0:60; ˙0:37/ D .˙0:60; ˙0:37; 0:37; 0:60/ D .˙0:60; 0:37; 0:37; ˙0:60/ D .˙0:37; 0:60; ˙0:60; 0:37/:

The eigenvectors of the adjacency matrix are equivalent to the eigenvectors of the Hückel secular matrix. Thus the eigenvectors correspond to the Hückel MOs of a molecule. Since the connection factor is given by C1, opposite to the negative value of the off-diagonal terms in the Hückel secular matrix, eigenvectors with the upper and lower signs in the eigen vectors correspond to bonding MOs and antibonding MOs, respectively.


131

Fig. 6.1 Vertex-signed graphs G43 (up) and b-G44 (down)

Fig. 6.2 Edge-signed graphs G43 (up) and b-G44 (down)

b-G44 If the number of edges is greater than or equal to the number of vertices in a connected graph, there must be a cyclic or ring structure in the graph. Methylenecyclopropane and methyl-cyclopropane are typically examples of b-G44 graph. The adjacency matrix of the b-G44 graph is built in the same way as in the case of G43 generating: 1 0 0 1 1 1 B1 0 1 0C C A.b-G44 / D B @ 1 1 0 0 A: 1 0 0 0 The resulting spectra and eigenvectors are: 1 2 3 4

D 2:17; D 0:31; D 1:00; D 1:48;

1 2 3 4

D .˙0:61; ˙0:52; ˙0:52; ˙0:61/ D .˙0:25; 0:37; 0:37; ˙0:82/ D .0:00; ˙0:71; 0:71; 0:00/ D .˙0:75; 0:30; 0:30; 0:51/:

The positive and negative eigenvalues have the same meaning as those of G43 .

1.2.2 Signed Graphs The intuitive net-sign approach proposed by Lee et al. [19] attempts to study the graphical features of MO graphs and is thus different from the analysis in the two earlier papers [31, 32]. For complete descriptions of the topological features of internal connectivities, both ESGs and VSGs are needed (Figs. 6.1 and 6.2).

132


The main chemical application of the net signs found initially is the rationalization of the ordering of molecular orbital energy levels (see below). The prediction of the stable ground-state geometry based on an energy-like quantity which is defined as the sum of the net signs over occupied MOs is found to agree satisfactorily with the predictions. The eigenvectors of molecular graphs coincide with the n-electron molecular orbitals in the tight-binding HMO model. Bearing this in mind, we immediately see that the net sign, (6.1), counts the chemical bonds intersected by, and the atoms lying on nodal planes of the respective molecular orbital. In particular, the larger the number of such chemical bonds and atoms, the smaller (in algebraic sense) is the corresponding net sign. The net-sign approach [19] can be viewed as quantification and mathematical reformulation of Wilson’s reasoning [29]. Therefore, it should by no means be surprising that in the great majority of cases the ordering of the MOs according to their energies coincides with the ordering according to their net signs. In other words, if we label the graph eigenvalues ina nonincreasing order: 1 2 n ;

(6.4)

then from the study of a large number of examples and some infinite series of graphs [24, 25, 33–36] we may expect that also S1 S2 S n :

(6.5)

Examples Compound

-Orbital

Energya

Net sign

Hexatriene

1au 1bg 2au

1.80 1.25 0.45

5 3 1

Butadiene

1au 1bg

1.62 0.62

3 1

3-Vinyl-hexatriene

1b1 2b1 1a2 3b1

2.03 1.32 1.00 0.37

7 5 2 1

2-Vinyl-butadiene

1b1 1a2 2b1

1.93 1.00 0.52

5 2 1

a

Values in this column are taken from [37] and are equal to eigenvalues of adjacency matrices of compounds

However, the relations (6.5) are not generally valid: a few examples when they are violated have been discovered. Furthermore, the net-sign approach encounters certain difficulties in the case of degenerate MOs; then an appropriate modification of (6.3) is necessary.


133

1.2.3 Degeneracy Degeneracy imposed difficulty on the original version of net-sign analysis using as an example the twofold degenerate 1b1g and la1u -orbitals of biphenyl [38]. Several other biphenyl-like graphs can be found [39]. In order to understand how the simple net-sign approach fails and how to solve the ambiguity raised by the degeneracy, let us consider the net sign of the degenerate -levels of p-biphenylmethyl. The eigenvalue of these twofold degenerate levels is 1.00. One might choose the eigenvectors as a pair of eigenvectors, X 0 i and X 00 i , which correspond to a pair of VSGs X 0 i ˛.1; 1; 0; 1; 1; 0; 0; 0; 0; 0; 0; 0; 0/; X 00 i ˛.0; 0; 0; 0; 0; 0; 1; 1; 0; 1; 1; 0; 0/ and localized on only one hexagon. They correspond to a pair of distinct ESGs with a net sign of two. An alternative is a pair of delocalized eigenvectors, X 0 j and X 00 j , which correspond to a pair of VSGs X 0 j˛.1; 1; 0; 1; 1; 0; 1; 1; 0; 1; 1; 0; 0/; X 00 j˛.1; 1; 0; 1; 1; 0; 1; 1; 0; 1; 1; 0; 0/; which span over two hexagons. They correspond to the same ESG with net sign of four. Thus, the simple net-sign approach fails to give a single net-sign value for the degenerate levels. Lee and Gutman [26] proposed a solution to overcome the ambiguity caused by the degeneracy in the net-sign approach. They defined a quantity, Srq , for any connecting pair of vertices (r; q) in an N-vertex molecular graph having d degenerate states as: d X ir iq (6.6) Srq D sign i D1

for the set of eigenvectors X D fij ji D 1; 2; : : : ; d I j D 1; 2; : : : ; N g: From X , a new set of eigenvectors Y D fYij ji D 1; 2; : : : ; d I j D 1; 2; : : : ; N g can be built by Y D Ux; where U is a unitary matrix. The quantity, Srq , is found to be independent of the choice of eigenvectors due to the following equality: Srq D sign

d X i D1

ir iq D sign

d X i D1

Yir Yiq

(6.7)

134


and thus the net sign of an ESG can be uniquely defined by: SD

X

Srq :

(6.8)

r n/:

(6.10)

kD1

Similarly the antibonding orbital information index, Ia is defined as the summation of the numerical values obtained via (6.9) for each molecular orbital level, k, ranging from 1 to N , provided the total number of positive (C) signs “m” should be less than the total number of negative () signs “n”: Ia D

N X

Ik .m < n/:

(6.11)

kD1

And the nonbonding orbital information index, In is defined as the summation of the numerical values obtained via (6.1) for each molecular orbital level, k, ranging from 1 to N , provided the total number of positive (C) signs “m” should be equal to the total number of negative () signs “n”: In D

N X kD1

Ik .m D n/:

(6.12)


137

And further investigation of these above [66] information-theoretic indices derived from ESGs has also been reported [67].

Definitions The new net-sign identity information index, I" , is defined as the summation of the square of the numerical values obtained for Ik [66] for each molecular orbital level, k, ranging from 1 to N : I" D

N X

ŒIk 2 :

(6.13)

kD1

From (6.9), Ik D m lb mn lb n.nm/ lbjnmj, where “m” and “n” represent the total number of positive (C) and total number of negative () signs from ESGs of the corresponding molecular graph and “k” is the molecular orbital level. On comparing with Wiener index (W ), Randic’s connectivity index (), and Balaban’s distance sum connectivity index (J ), the net-sign identity information index, I" , has been proved to be better toward predicting optical properties like molecular refraction for both saturated and unsaturated hydrocarbons. The new netsign information indices are intriguing in describing optical properties as compared to bulk properties in both saturated and unsaturated hydrocarbons. Moreover, the p square root of the bonding information index, Ib , is also found to be potential and much better, as compared to Randic’s connectivity index () for bulk properties p (MV, BP, etc.), and the square root of net-sign identity information index, I" , is even found better than the novel net-sign identity information index, I" , for optical properties like MR. More detail investigations have been examined and mentioned in Refs. [66, 67]. The present work enunciates the graph (molecule) signature of non-alternant non-benzenoid hydrocarbons with odd member of rings (non-bipartite molecular graphs) based on chemical signed graph theory. Experimental evidences and Hückel spectrum reveal that the conjugated hydrocarbons possessing nonbonding molecular orbitals (NBMOs) is very unstable and highly reactive even under drastic conditions of low temperature [68,69]. The problem of investigation of the energy level patterns was imposed in a series of papers by Bochvar et al. [70–72], but no general results have been obtained [73]. The graph theoretical approach to conjugated hydrocarbons, using simple Hückel MO calculation in a series of paper [74–80], justifies the obtained results qualitatively. Earlier Gutman [81] proposed the enumeration of the number of NBMOs and bonding orbitals from the characteristic polynomial [82] of the molecular graph of conjugated systems. We have enunciated these magnitudes of numbers on the basis of chemical signed graph theory, as net-sign approach provides a convenient way to compute the number of NBMOs along with bonding orbitals and anti-bonding orbitals. We have also compared our

138


predictions via net-sign approach with those obtained using Hückel calculations. Moreover, classifications of such conjugated hydrocarbons are carried out based on their spectral characteristic. Earlier, the conjugated circuit model, proposed by Randic [83–86], provides a basis for classification of conjugated system based on aromaticity and anti-aromaticity criteria. Recently, Balaban and Randic [87] have provided a comprehensive bibliography for non-alternant conjugated hydrocarbons, including both experimental and theoretical available data. Due to the peculiarity in its chemical properties and behaviors, non-bipartite molecular graphs are taken into consideration for the present study. The aromaticity and anti-aromaticity criteria is beyond our scope in this present study.

2 Non-bipartite Molecular Graphs 2.1 Non-benzenoid Systems with Odd Rings Having Only 4n C 2 Conjugated Circuits Figure 6.3 represents the non-bipartite molecular graphs of 15 non-benzenoid systems, including azulene (1/1), acepleiadylene (8/1), and corannulene (15/1) having only 4n C 2 conjugated circuits. Net signs of ESGs and energy levels of HMO’s for these systems are listed in Table 6.1. The ordering according to the net sign is found not in perfect agreement with the ordering to the energy levels. Table 6.2 listed the graph (molecule) signature of these systems along with their classifications on the basis of spectral characteristics derived through net-sign approach. It is interesting to note that most of these systems are found to be stable. As a comparison to Randic’s conjugated circuit model, the results obtained for these systems also support the definition, 4n C 2 conjugated circuits are stable and aromatic, though they posses odd-membered ring. The aromaticity criterion is beyond our scope in this present study. However, molecules such as (3/1), (5/1), (6/1), and (12/1) are observed as reactive. Moreover, iso-conjugated molecules [83], though possessing same RE, still are characterized to be different in properties. As we can see for the iso-conjugated pair, (2/1) and (3/1), the later is observed to be reactive (EE), as compared to the stable former. It is expected that the central ring size must have important role in determining the properties of the molecule. Such different molecular properties can also be observed for other iso-conjugated molecules. In addition to ring size, other factors (e.g. steric) must contribute to the overall structure. In Table 6.2, we have also compared our qualitative predictions for the nonbenzenoid hydrocarbons having only 4n C 2 conjugated circuits with those obtained by Hückel calculations. It is interesting to note that no similar results are obtained for systems such as (3/1), (5/1), (6/1), and (12/1). It is claimed that net-sign approach is derived from molecular electronic structure instead of atomic structure and hence justified its reliability and prediction as compared to prediction made using Hückel


139

Fig. 6.3 Non-bipartite molecular graphs of non-benzenoid systems with odd member of rings having only 4n C 2 conjugated circuits

140


Table 6.1 Net signs of ESGs and energy levels of HMOs for non-benzenoid systems with odd member of rings having only 4n C 2 conjugated circuits Molecule 1/1 2/1

3/1

4/1

5/1

6/1

7/1

8/1

9/1

10/1

11/1

12/1

13/1

14/1

15/1

Energy 2.095, 1.869, 1.579, 0.737, 0.400, 0.477, 0.887, 1.356, 1.652, 2.310 2.261, 2.093, 1.683, 1.387, 1.147, 0.653, 0.316, 0.422, 0.818, 1, 1.339, 1.497, 2.069, 2.396 2.251, 2.076, 1.797, 1.321, 1.046, 0.668, 0.388, 0.323, 0.842, 1.114, 1.396, 1.507, 1.903, 2.461 2.335, 1.838, 1.714, 1.683, 1, 0.661, 0.215, 0.261, 0.840, 1, 1.449, 1.476, 2, 2.420 2.221, 2.112, 2.053, 1.69, 1.628, 0.949, 0.372, 0.227, 0.441, 0.521, 0.912, 1, 1, 1.286, 1.532, 1.882, 2.243, 2.434 2.259, 2.051, 1.974, 1.802, 1.732, 0.808, 0.776, 0.445, 0.408, 0.216, 0.535, 1, 1.247, 1.438, 1.443, 1.732, 2.091, 2.552 2.247, 1.879, 1.784, 1.651, 1.218, 0.555, 0, 0.347, 0.802, 0.858, 1.274, 1.532, 2.144, 2.377 2.451, 2, 1.802, 1.678, 1.247, 1, 0.503, 0.445, 0.445, 0.668, 1, 1.247, 1.423, 1.802, 2, 2.540 2.412, 2.098, 1.869, 1.463, 1.273, 1.078, 0.504, 0.405, 0.329, 0.827, 0.917, 1.251, 1.468, 1.803, 1.955, 2.552 2.440, 2.029, 1.856, 1.577, 1.266, 0.932, 0.729, 0.281, 0.411, 0.651, 1.256, 1.273,1.804, 2.034, 2.533 2.363, 2.218, 1.806, 1.330, 1.267, 1.211, 0.667, 0.132, 0.176, 0.796, 1.253, 1.345, 1.809, 1.989, 2.546 2.521, 2.141, 1.932, 1.732, 1.589, 1.194, 1, 1, 0.673, 0.198, 0.454, 0.586, 1, 1, 1.117, 1.426, 1.732, 1.871, 2.232, 2.562 2.511, 2.146, 1.929, 1.719, 1.638, 1.129, 1.119, 1, 0.396, 0.369, 0.316, 0.816, 0.847, 1, 1.156, 1.384, 1.809, 1.835, 2.243, 2.549 2.491, 2.205, 1.906, 1.732, 1.650, 1.078, 1, 1, 0.523, 0.399, 0.213, 0.808, 1, 1, 1.28, 1.372, 1.732, 1.793, 2.177, 2.609 2.467, 2.467, 1.777, 1.777, 1.356, 1.356, 1.214, 1, 0.477, 0.477, 0.737, 0.737, 0.737, 0.777, 0.777, 1.467, 1.467, 1.539, 2.095, 2.095, 2.675

Net sign 7, 9, 5, 3, 1, 1, 3, 5, 7, 11 12, 10, 10, 8, 6, 4, 2, 2, 4, 4, 6, 8, 12, 16 12, 12, 10, 6, 4, 4, 2, 0, 4, 6, 8, 8, 12, 16, 12, 16 12, 12, 8, 8, 4, 4, 0, 0, 4, 5, 8, 8, 12, 16 13, 11, 10, 10, 7, 7, 7, 3, 0, 2, 1, 5, 4, 8, 11, 11, 16, 21 13, 17, 12, 10, 11, 7, 4, 2, 1, 0, 1, 6, 6, 11, 8, 11, 16, 21 12, 8, 12, 8, 4, 4, 0, 0, 4, 4, 8, 8, 12, 16 15, 14, 10, 9, 6, 1, 3, 2, 2, 5, 3, 2, 7, 10, 13, 19, 10, 13, 19 15, 13, 13, 9, 9, 7, 5, 3, 3, 5, 5, 5, 9, 11, 13, 19 15, 13, 11, 9, 5, 3, 5, 1, 3, 5, 5, 7, 9, 11, 13, 19 15, 17, 9, 7, 5, 7, 5, 1, 1, 5, 7, 5, 7, 11, 13, 19 20, 16, 16, 12, 12, 10, 5, 7, 6, 0, 4, 4, 6, 6, 6, 10, 12, 14, 18, 24 22, 15, 16, 11, 10, 6, 7, 3, 4, 3, 1, 5, 6, 3, 8, 9, 14, 13, 16, 24 22, 15, 16, 12, 11, 8, 7, 7, 3, 4, 1, 6, 7, 7, 8, 9, 12, 13, 16, 24 20, 21, 13, 13, 11, 12, 5, 5, 5, 5, 5, 5, 4, 5, 9, 9, 15, 17, 17, 25


141

Table 6.2 Graph (molecule) signature of non-bipartite molecular graphs with odd member of rings having only 4n C 2 conjugated circuits and their classification Molecule 1/1 2/1 3/1 4/1 5/1 6/1 7/1 8/1 9/1 10/1 11/1 12/1 13/1 14/1 15/1

¢a 0 0 1 0 1 1 0 0 0 0 0 1 0 0 0

¢b 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

N0c 0 0 1 2 1 1 2 0 0 0 0 1 0 0 0

N0d 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

Typee S S S S S S S S S S S S S S S

Typef S S EE PR ED EE PR S S S S ED S S S

Typeg S S S S S S ED S S S S S S S S

S stable, PR poly radical, EE electron excessive, ED electron deficient a Graph (molecule) signature based on net-sign approach b Graph (molecule) signature based on Hückel calculation c NBMOs obtained via net-sign approach d NBMOs obtained via Hückel calculation e Stable (aromatic) as per conjugated circuit model [86] f Spectral characterizations and their classifications [1, 2] based on net-sign approach g Spectral characterizations and their classifications [1, 2] based on Hückel calculation

calculations. For all those systems, (3/1), (5/1), (6/1), and (12/1) found as reactive (ED or EE), it is believed that the nonbonding orbitals predicted via net-sign approach may play the key role toward their instability. Moreover, the system (4/1) and (7/1), which is predicted to be PR stable via net-sign approach, is found to be stable and reactive (ED), respectively, via Hückel calculations.

2.2 Non-benzenoid Systems with Odd Rings Having 4n C 2 and 4 n Conjugated Circuits Figure 6.4 represents the non-bipartite molecular graphs of ten non-benzenoid systems. Among them, aceheptylene (1/2), azupyrene (2/2), aceazulene (3/2), dicycloheptapentalene (4/2), azulenoheptalene (5/2), as-azupyrene (6/2), and pyracylene (7/2) posses prevailing contribution from 4n C 2 and 4n conjugated circuits. Net signs of ESGs and energy levels of HMOs for these systems are listed in Table 6.3. The ordering according to the net sign is found not in perfect agreement with the ordering to the energy levels. Table 6.4 listed the graph (molecule) signature of these systems along with their classifications on the basis of spectral characteristics derived through net-sign

142


Fig. 6.4 Non-bipartite molecular graphs of non-benzenoid systems with odd member of rings having 4n C 2 and 4n conjugated circuits

approach. It can be observed that among these intermediate classes of molecules, graphs of conjugated systems with prevailing contribution from 4n C 2 circuits are found to be stable, except azulenoheptalene (5/2). On the other hand, graphs of conjugated systems with prevailing contribution from 4n circuits are found to be reactive.


143

Table 6.3 Net signs of ESGs and energy levels of HMOs for non-benzenoid systems with odd member of rings having 4n C 2 and 4n conjugated circuits Molecule 1/2

2/2

3/2 4/2

5/2

6/2

7/2 8/2

9/2

10/2

Energy 2.242, 1.942, 1.933, 1.497, 0.803, 0.709, 0.329, 0.241, 0.758, 1.136, 1.321, 1.771, 1.781, 2.447 2.290, 2, 2, 1.802, 1.247, 0.879, 0.445, 0.362, 0.445, 0.583, 1, 1.247, 1.347, 1.802, 2.069, 2.532 2.175, 1.919, 1.901, 1.310, 0.594, 0.284, 0.480, 0.831, 1, 1.682, 1.708, 2.481 2.199, 2.130, 2, 1.802, 1.247, 0.713, 0.478, 0.445, 0.445, 0.494, 1, 1.247, 1.544, 1.802, 1.912, 2.569 2.296, 2.111, 2, 1.829, 1.449, 0.818, 0.779, 0.702, 0.215, 0.133, 0.464, 1.094, 1.158, 1.427, 1.551, 1.851, 1.992, 2.530 2.268, 2.046, 2, 1.775, 1.300, 0.79, 0.488, 0.307, 0.303, 0.628, 1, 1.260, 1.432, 1.807, 1.989, 2.554 2.414, 2, 1.732, 1.562, 1, 1, 0, 0.414, 1, 1, 1, 1.732, 2, 2.561 2.267, 2.217, 1.955, 1.396, 1.221, 1.073, 0.847, 0.116, 0.544, 0.598, 1.124, 1.152, 1.376, 1.627, 2.114, 2.557 2.319, 2.101, 2.026, 1.373, 1.259, 1, 0.923, 0, 0.296, 0.811, 1, 1.259, 1.322, 1.652, 2.101, 2.561 2.368, 2.212, 2.180, 1.577, 1.446, 1.315, 1, 1, 0.826, 0.038, 0.372, 0.595, 1, 1, 1.234, 1.467, 1.492, 1.847, 2.303, 2.577

Net sign 10, 11, 12, 7, 4, 3, 2, 1, 4, 5, 8, 9, 10, 16 11, 13, 13, 10, 6, 5, 2, 3, 2, 5, 3, 6, 11, 10, 13, 19 8, 9, 8, 5, 4, 1, 2, 3, 4, 7, 6, 14 11, 13, 14, 10, 6, 3, 5, 2, 2, 3, 5, 6, 11, 10, 13, 19 13, 15, 16, 13, 7, 5, 3, 5, 3, 1, 1, 7, 9, 7, 13, 13, 15, 21 11, 13, 14, 11, 7, 5, 3, 1, 1, 3, 4, 7, 9, 11, 13, 19 13, 12, 8, 7, 5, 5, 0, 3, 5, 5, 5, 8, 11, 17 17, 13, 9, 9, 5, 5, 1, 1, 1, 3, 7, 3, 9, 5, 11, 19 15, 14, 11, 9, 6, 4, 3, 0, 1, 3, 4, 6, 9, 11, 14, 19 20, 16, 20, 12, 12, 8, 6, 6, 4, 0, 4, 4, 6, 6, 8, 12, 12, 16, 20, 24

In Table 6.4, as compared to the prediction made by Hückel calculation, it can be observed that the system (5/2) is found to be ED via net-sign approach, whereas Hückel calculation predicted it to be stable. In contrast to the prediction made to systems, (7/2) and (9/2) via net-sign approach as ED and EE, Hückel calculations resulted as EE and ED, respectively.

2.3 Non-benzenoid Systems with Odd Rings Having Only 4n Conjugated Circuits Figure 6.5 represents the non-bipartite molecular graphs of 19 non-benzenoid systems, including pentalene (1/3), heptalene (2/3), and s-indacene (3/3) having only 4n conjugated circuits. Net signs of ESGs and energy levels of HMOs for

144


Table 6.4 Graph (molecule) signature of non-bipartite molecular graphs with odd member of rings having 4n C 2 and 4n conjugated circuits and their classification Molecule 1/2 2/2 3/2 4/2 5/2 6/2 7/2 8/2 9/2 10/2

¢a 0 0 0 0 2 0 1 0 1 1

¢b 0 0 0 0 0 0 1 0 1 2

N0c 0 0 0 0 0 0 1 0 1 1

N0d 0 0 0 0 0 0 1 0 1 0

Typee S S S S S US US US US US

Typef S S S S EE S ED S EE ED

Typeg S S S S S S EE S ED ED

S stable, PR poly radical, EE electron excessive, ED electron deficient a Graph (molecule) signature based on net-sign approach b Graph (molecule) signature based on Hückel calculation c NBMOs obtained via net-sign approach d NBMOs obtained via Hückel calculation e Stable (aromatic) and unstable (anti-aromatic) as per conjugated circuit model [86] f Spectral characterizations and their classifications [1, 2] based on net-sign approach g Spectral characterizations and their classifications [1, 2] based on Hückel calculation

these systems are listed in Table 6.5. The ordering according to the net sign is found not in perfect agreement with the ordering to the energy levels. Table 6.6 listed the graph (molecule) signature of these systems along with their classifications on the basis of spectral characteristics derived through net-sign approach. It is interesting to note that most of these systems are found to be unstable (EE or ED), supporting the definition of 4n-conjugated circuits, as a comparison. However, few exceptional cases are observed, which may be due to Jahn–Teller distortion [88], reducing the contribution of 4n-conjugated circuits [86]. In Table 6.6, it can be observed that the prediction made via net-sign approach for the spectral classifications of non-benzenoid hydrocarbons having only 4n-conjugated circuits are not same with those obtained through Hückel calculations. The systems (4/3) and (17/3) are predicted to be ED through Hückel calculations in contrast to EE via net-sign approach. The systems such as (10/3), (12/3), (13/3), (15/3), and (16/3) are found to be reactive through Hückel calculations as compared to those as stable via net-sign approach. Moreover, the system (19/3) is predicted to be PR stable via net-sign approach in contrast to ED through Hückel calculations.

3 Summary Due to peculiarity in the chemical properties and behaviors, non-bipartite molecular graphs (non-alternant non-benzenoid PAHs having odd member of rings) are taken into consideration for the systematic study toward obtaining graph (molecule)


145

Fig. 6.5 Non-bipartite molecular graphs of non-benzenoid systems with odd member of rings having only 4n conjugated circuits

146


Fig. 6.5 (continued) Table 6.5 Net signs of ESGs and energy levels of HMOs for non-benzenoid systems with odd member of rings having only 4n conjugated circuits Molecule

Energy

Net sign

1/3 2/3

2, 1.813, 1.414, 0, 0.471, 1, 1.414, 2.343 2.170, 1.891, 1.732, 1, 0.705, 0.311, 0, 1, 1.317, 1.481, 1.732, 2.278 2.303, 1.757, 1.618, 1.618, 0.820, 0, 0.618, 0.618, 1.139, 1.303, 2.000, 2.438 2.319, 1.756, 1.699, 1.414, 1, 0.239, 0.304, 0.832, 1, 1.414, 1.938, 2.460 2.450, 1.828, 1.639, 1.261, 0.584, 0.339, 0.375, 0.714, 1.324, 1.335, 1.812, 2.540 2.278, 2.245, 1.732, 1.317, 1.132, 1, 0, 0.456, 0.705, 1, 1.335, 1.732, 1.891, 2.586 2.362, 1.854, 1.802, 1.802, 1.133, 0.826, 0.445, 0.445, 0, 0.679, 1.247, 1.247, 1.508, 1.586, 2, 2.400 2.381, 1.854, 1.838, 1.683, 1.248, 0.710, 0.618, 0.215, 0.181, 0.840, 1.068, 1.347, 1.476, 1.618, 1.959, 2.420 2.414, 2, 1.732, 1.675, 1.414, 1.170, 0.539, 0, 0.414, 0.689, 1, 1, 1.414, 1.732, 2.214, 2.481 2.448, 1.945, 1.732, 1.618, 1.515, 0.926, 0.896, 0.208, 0.421, 0.618, 0.843, 1.268, 1.315, 1.732, 2.151, 2.523 2.431, 2.191, 1.848, 1.414, 1.414, 1.327, 0.765, 0.577, 0.301, 0, 0.765, 1, 1.264, 1.414, 1.414, 1.848, 2.059, 2.503 2.430, 1.979, 1.729, 1.677, 1.414, 1.105, 0.702, 0.161, 0.356, 0.696, 0.869, 1.138, 1.414, 1.705, 2.195, 2.499

5, 7, 4, 0, 1, 3, 4, 9 9, 11, 8, 4, 3, 1, 0, 4, 5, 7, 8, 13 10, 10, 6, 6, 2, 0, 2, 2, 6, 6, 10, 14 9, 9, 6, 6, 4, 2, 2, 2, 3, 6, 10, 14 10, 10, 6, 6, 2, 2, 2, 2, 4, 6, 10, 14 13, 15, 8, 5, 7, 4, 0, 1, 3, 4, 9, 8, 11, 17 13, 13, 9, 9, 5, 7, 1, 1, 0, 3, 7, 5, 9, 11, 13, 17

3/3 4/3 5/3 6/3 7/3

8/3

9/3 10/3

11/3

12/3

14, 14, 10, 10, 6, 2, 3, 2, 2, 6, 6, 6, 10, 9, 14, 18 15, 10, 13, 10, 7, 5, 2, 0, 1, 3, 6, 6, 9, 11, 14, 19 15, 10, 13, 6, 7, 3, 6, 1, 2, 2, 5, 6, 7, 11, 14, 19 17, 19, 12, 8, 8, 9, 4, 3, 1, 0, 4, 7, 5, 8, 8, 12, 15, 21 15, 9, 13, 11, 7, 5, 3, 1, 1, 3, 5, 6, 9, 11, 13, 19

(continued)


147

Table 6.5 (continued) Molecule

Energy

Net sign

13/3

2.443, 1.965, 1.717, 1.703, 1.286, 1.190, 0.776, 0.203, 0.488, 0.529, 0.9, 1.141, 1.497, 1.622, 2.186, 2.515 2.441, 2.095, 1.839, 1.822, 1.567, 1.322, 0.846, 0.737, 0.515, 0.308, 0, 0.477, 1, 1.195, 1.356, 1.488, 1.523, 1.816, 2.179, 2.459 2.458, 2.074, 1.839, 1.822, 1.552, 1.296, 0.967, 0.644, 0.526, 0.268, 0.127, 0.624, 0.939, 1.197, 1.353, 1.472, 1.551, 1.798, 2.162, 2.476 2.473, 2.061, 1.834, 1.831, 1.414, 1.414, 1, 0.656, 0.396, 0.378, 0.156, 0.694, 1, 1.094, 1.414, 1.414, 1.591, 1.764, 2.153, 2.491 2.086, 1.879, 1.879, 1, 0.347, 0.347, 0.572, 1.532, 1.532, 2.514 2.414, 1.947, 1.912, 1.802, 1.503, 1.247, 0.445, 0.414, 0.445, 0.489, 0.713, 1.247, 1.324, 1.802, 2.199, 2.637 2.561, 2.066, 2, 2, 1.414, 1, 1, 1, 0.222, 0, 0, 0, 1, 1, 1.414, 1.561, 1.652, 2, 2, 2.636

15, 9, 13, 11, 9, 7, 3, 1, 3, 1, 5, 7, 9, 11, 13, 19

14/3

15/3

16/3

17/3 18/3

19/3

19, 14, 17, 14, 11, 9, 6, 6, 3, 1, 0, 2, 7, 7, 10, 10, 13, 15, 18, 23 19, 13, 17, 15, 11, 7, 5, 5, 3, 1, 1, 3, 7, 7, 9, 11, 11, 15, 17, 23 19, 13, 17, 15, 9, 9, 7, 5, 1, 3, 1, 3, 7, 9, 9, 11, 13, 11, 17, 23 6, 7, 7, 3, 1, 1, 0, 5, 5, 12 12, 8, 12, 10, 8, 6, 2, 0, 2, 4, 4, 6, 8, 10, 16, 20 16, 16, 16, 16, 8, 8, 4, 4, 0, 0, 0, 0, 8, 8, 8, 8, 16, 16, 16, 24

Table 6.6 Graph (molecule) signature of non-bipartite molecular graphs with odd member of rings having only 4n conjugated circuits and their classification Molecule

¢a

¢b

1/3 2/3 3/3 4/3 5/3 6/3 7/3 8/3 9/3 10/3 11/3 12/3 13/3 14/3 15/3 16/3 17/3 18/3 19/3

1 1 1 4 0 1 1 2 1 0 1 0 0 1 0 0 1 1 0

1 1 1 2 0 1 1 2 1 2 1 2 2 1 2 2 2 2 1

N0c

N0d

Typee

Typef

Typeg

1 1 1 0 0 1 1 0 1 0 1 0 0 1 0 0 1 1 4

1 1 1 0 0 1 1 0 1 0 1 0 0 1 0 0 0 0 3

US US US US US US US US US US US US US – – – US US US

ED EE ED EE S ED EE EE ED S EE S S EE S S EE ED PR

ED EE ED ED S ED EE EE ED ED EE ED ED EE EE EE ED ED EE

S stable, PR poly radical, EE electron excessive, ED electron deficient a Graph (molecule) signature based on net-sign approach b Graph (molecule) signature based on Hückel calculation c NBMOs obtained via net-sign approach d NBMOs obtained via Hückel calculation e Unstable (anti-aromatic) as per conjugated circuit model [86] f Spectral characterizations and their classifications [1, 2] based on net-sign approach g Spectral characterizations and their classifications [1, 2] based on Hückel calculation

148


signatures based on chemical signed graph theory. Moreover, classifications of such conjugated hydrocarbons are carried out based on their spectral characteristic and are analyzed with the conjugated circuit model, as a comparison. It is also interesting to note that no similar predictions can be made using simple Hückel calculations. Benzocyclobutadiene derivatives are beyond the scope of the present study. We agree with the fact that though the rationalization of the ordering of molecular orbital energy levels are not observed for all the non-bipartite graphs, still the net-sign approach gives a distinct way to classify the non-bipartite graphs combining with the number of NBMOs. It is believed that net-sign approach is derived from molecular electronic structure instead of atomic structure and hence justified its reliability and predictions as compared to predictions made using simple Hückel calculations. It would be of interest to see whether any of these structures can be synthesized and to what extent this prediction will be satisfied. Acknowledgments We are thankful to Milan Randic for his suggestions. This research is supported by National Science Council, Taiwan.

References 1. Prelog, V.: Nobel lecture (December 1975). reprinted in Science 193, 17 (1976) 2. Cyvin, S.J.: The number of Kekule structures for primitive coronoids(cycloarenes). Chem. Phys. Lett. 147, 384–388 (1988) 3. Aboav, D., Gutman, I.: Estimation of the number of benze-noid hydrocarbons. Chem. Phys. Lett. 148, 90–92 (1988) 4. Kier, L.B., Hall, L.H.: Molecular Connectivity in Chemistry and Drug Research. Academic Press, New York (1976) 5. Trinajstii, N.: Chemical Graph Theory, Chap. 4, Vol. 2. CRC Press, Boca Raton, FL (1983) 6. Kier, L.B., Hall, L.H.: Molecular Connectivity in Structure-Activity Analysis. Wiley, New York (1986) 7. Read, R.C., BaIaban, A.T. (eds.): Chemical Applications of Graph Theory, Chap. 4. Academic Press, New York (1976) 8. Balaban, A.T., BaIaban, A.T. (eds.): Chemical Applications of Graph Theory, Chap. 5. Academic Press, New York (1976) 9. Gielen, M., BaIaban, A.T. (eds.): Chemical Applications of Graph Theory, Chap. 9. Academic Press, New York (1976) 10. King, R.B., Rouvray, D.H.: Chemical applications of topology and group theory. Theoret. Chim. Acta 69, 1–10 (1986) 11. King, R.B.: Chemical bonding topology of bare post-transition-metal clusters: analogies between condensed-phase and gas-phase species. J. Phys. Chem. 92, 4452–4456 (1988) 12. El-Basil, S.: Combinatorial Clar sextet theory: on valence-bond method of Herndon and Hosoya. Theor. Chim. Acta 70, 53–65 (1986) 13. Cyvin, S.J., Cyvin, B.N., Brunvoll, J.: Half essentially disconnected coronoid hydrocarbons. Chem. Phys. Lett. 140, 124–129 (1987) 14. Arteca, G.A., Mezey, P.G.: A topological characterization for simple molecular surfaces. J. Mol. Struct. (Theochem) 166, 11–16 (1988) 15. Fernandez, A.: Pattern of separatrices and intrinsic reaction coordinates for degenerate thermal rearrangements. Theor. Chim. Acta 67, 229–233 (1985)


149

16. Sakamoto, A., Kawakami, H., Yoshikawa, K.: A graph theoretical approach to complex reaction networks. Chem. Phys. Lett. 146, 444–448 (1988) 17. Randic, M.: Symmetry properties of chemical graphs. VIII. On complementarity of isomerization modes. Theor. Chim. Acta 67, 137–155 (1985) 18. King, R.B., Reich1, L.E., Schieve, W.C. (eds.): Instabilities, Bifurcations, and Fluctuations in Chemical Systems, p. 47. University of Texas Press, Austin (1982) 19. Lee, S.L., Lucchese, R.R., Chu, S.Y.: Topological analysis of eigenvectors of the adjacency matrices in graph theory: The concept of internal connectivity. Chem. Phys. Lett. 137, 279 (1987) 20. Gunthard, H.H., Primas, H.: Zusammenhang von Graphentheorie und MO-Theorie von Molekeln mit Systemen konjugierter Bindungen. Helv. Chim. Acta 39, 1645–1653 (1956) 21. Trinajstic, N., Segal, G.A. (eds.): Semiempirical Methods of Electronic Structure Calculation. Part A. Techniques, Vol. 7, p. 1. Plenum Press, New York (1977) 22. Ruedenburg, K.: Quantum mechanics of mobile electrons in conjugated bond systems. I. General analysis in the tight-binding formulation. J. Chem. Phys. 34, 1861–1877 (1961) 23. Schmidtke, H.H.: LCAO description of symmetric molecules by unified theory of finite graphs. J. Chem. Phys. 45, 3920–3928 (1966) 24. Lee, S.L., Li, F.Y.: Net sign approach in graph spectral theory. J. Mol. Struct. (Theochem) 207, 301–317 (1991) 25. Lee, S.L., Li, C.: Chemical signed graph theory. Int. J. Quant. Chem. 49, 639–648 (1994) 26. Lee, S.L., Gutman, I.: Topological analysis of the eigenvectors of the adjacency matrices in graph theory: Degenerate case. Chem. Phys. Lett. 157, 229–232 (1989) 27. Landau, L.D., Lifshitz, E.M.: Quantum Mechanics: Non-relativistic Theory, Course of Theoretical Physics, 3rd edn., Vol. 3, p. 60. Pergamon Press, New York (1977) 28. Levine, I.N.: Quantum Chemistry. Allyn and Bacon, Boston, MA (1974) 29. Wilson, E.B.: Symmetry, nodal surfaces, and energy ordering of molecular orbitals. J. Chem. Phys. 63, 4870–4879 (1975) 30. Lee, S.L., Li, F.Y., Lin, F.: Topological analysis of eigenvalues of particle in one- and twodimensional simple quantal systems: net sign approach. Int. J. Quant. Chem. 39, 59–70 (1991) 31. Heilbronner, E., Straub, P.A.: Slide rule computation of Hueckel molecular orbitals. Tetrahedron 23, 845–862 (1967) 32. Herndon, W.C., Silber, E.: Simplified molecular orbitals for organic molecules. J. Chem. Educ. 48, 502–508 (1971) 33. Lee, S.L., Yeh, Y.N.: Topological analysis of some special of graphs: Hypercubes. Chem. Phys. Lett. 171, 385–388 (1990) 34. Lee, S.L., Luo, Y.L., Sagan, E.B.: Eigenvector and eigenvalues of some special graphs. IV. Multilevel circulants, Yeh. Int. J. Quant. Chem. 41, 105–116 (1992) 35. Lee, S.L., Yeh, Y.N.: On Eigenvalues and Eigenvactors of graphs. J. Math. Chem. 12, 121–135 (1993) 36. Gutman, I., Lee, S.L., Yeh, Y.N.: Net signs of molecular graphs: dependence on molecular structure. Int. J. Quant. Chem. 49, 87–95 (1994) 37. Dehmer, M.: Information theoretic concepts for the analysis of complex networks. Appl. Artif. Intell. 22, 684–706 (2008) 38. Gutman, I.: Topological analysis of Eigenvalues of the adjacency matrices in graph theory: A difficulty with the concept of internal connectivity. Chem. Phys. Lett. 148, 93–94 (1988) 39. Coulson, C.A., Streitwieser, A.: Dictionary of -electron calculations. Freeman, San Francisco, CA (1965) 40. Lee, S.L.: Topological analysis of five-vertex clusters of group IVA elements. Theo. Chim. Acta. 81, 185–199 (1992) 41. Gutman, I., Lee, S.L., Yeh, Y.N.: Net signs and Eigenvalues of molecular graphs: some analogies. Chem. Phys. Lett. 191, 87–91 (1992) 42. Wiener, H.: Structural determination of paraffin boiling points. J. Am. Chem. Soc. 69, 17–20 (1947)

150


43. Wiener, H.: Influence of interatomic forces on paraffin properties. J. Chem. Phys. 15, 766 (1947) 44. Morowitz, H.: Some order-disorder considerations in living systems. Bull. Math. Biophys. 17, 81–86 (1953) 45. Quastler, H.: Information Theory in Biology. University of Illinois Press, Urbana (1953) 46. Dancoff, S.M., Quastler, H.: Information content and error rate of living things. In: Quastler, H. (eds.) Essays on the Use of Information Theory in Biology, pp. 263–274. University of Illinois Press, Urbana (1953) 47. Linshitz, H.: The information content of a battery cell. In: Quastler, H. (eds.) Essays on the Use of Information Theory in Biology. University of Illinois Press, Urbana (1953) 48. Mowshowitz, A.: Entropy and the complexity of graphs i: An index of the relative complexity of a graph. Bull. Math. Biophys. 30, 175–204 (1968) 49. Mowshowitz, A.: The information content of digraphs and infinite graphs. Bull. Math. Biophys. 30, 225–240 (1968) 50. Mowshowitz, A.: Graphs with prescribed information content. Bull. Math. Biophys. 30, 387– 414 (1968) 51. Mowshowitz, A.: Entropy measures and graphical structure. Bull. Math. Biophys. 30, 533–546 (1968) 52. Rashewsky, N.: Life, information theory, and topology. Bull. Math. Biophys. 17, 229–235 (1955) 53. Balaban, A.T.: Highly discriminating distance-based topological index. Chem. Phys. Lett. 89, 399–404 (1982) 54. Randic’, M.: On characterization of molecular branching. J. Am. Chem. Soc. 97, 6609–6615 (1975) 55. Kier, L.B., Hall, L.H., Murray, W.J., Randic’, M.: Molecular connectivity I: Relationship to nonspecific local anesthesia. J. Pharm. Sci. 64, 1971–1974 (1975) 56. Bonchev, D., Trinajstic, N.: On topological characterization of molecular branching. Int. J. Quant. Chem. Symp. 12, 293–303 (1978) 57. Bonchev, D., Balaban, A.T., Mekenyan, A.: Generalization of the graph center concept, and derived topological centric indexes. J. Chem. Inf. Comput. Sci. 20, 106–113 (1980) 58. Gutman, I., Ruscic, B., Trinajstic, N., Wilcox, C.F. Jr.: Graph theory and molecular orbitals. XII. Acyclic polyenes. J. Chem. Phys. 62, 3399–3405 (1975) 59. Hosoya, H.: Rydberg orbitals. IV. Basic formulas for the one-electron perturbation calculation of molecular Rydberg excited states. Int. J. Quant. Chem. 6, 801–817 (1972) 60. Basak, S.C., Gute, B.D., Grunwald, G.D.: A comparative study of topological and geometrical parameters in estimating normal boiling point and octanol/water partition coefficient. J. Chem. Inf. Comput. Sci. 36, 1054–1060 (1996) 61. Bonchev, D.: “Information Theoretic Indices for Characterization of Chemical Structure”. Wiley InterScience, New York (1983) 62. Balaban, A.T. (ed.): “From Chemical Topology to 3D Molecular Geometry”. Plenum Press, New York (1997) 63. Kubinyi, H., Folkers, G., Martin, Y.C. (eds.): “3D QSAR in Drug Design”, Vol. 1, 2, 3. Kluwer/ESCOM (1996–98) 64. Karelson, M.: “Molecular Descriptors in QSAR/QSPR”. Wiley-Interscience, New York (2000) 65. Todeschini, R., Consonni, V.: “Handbook of Molecular Descriptors”. Wiley-VCH, Weinheim, Germany (2000) 66. Sahu, P.K., Lee, S.L.: Novel information theoretic topological index Ik for unsaturated hydrocarbons. Chem. Phys. Lett. 396, 465–468 (2004) 67. Sahu, P.K., Lee, S.L.: Net-sign identity information index: a novel approach towards numerical characterization of chemical signed graph theory. Chem. Phys. Lett. 454, 133–138 (2008) 68. Lloyd, D.: The chemistry of cyclic conjugated compounds: to be or not to be like benzene? Wiley, Chichester (1989) 69. Lin, C.Y., Krantz, A.: Matrix preparation of cyclobutadiene. J. Chem. Soc. Chem. Commun. 1111–1112 (1972)


151

70. Bochvar, D.A., Stankevich, I.V., Tutkevich, A.V.: izv akad nauk ussr. Seria Khim 1185 (1969) 71. Bochvar, D.A., Gal’pern, E.G.: izv akad nauk ussr. Seria Khim 1327 (1972) 72. Bochvar, D.A., Stankevich, I.V.: Zh Strukt Khim 13, 1123 (1972) 73. Gutman, I., Trinajstic, N.: Graph-theoretical classifications of conjugated hydrocarbons. Naturwissenschaften 60, 475–475 (1973) 74. Groavac, A., Gutman, I., Trinajstic, N., Zivkovic, T.: Graph theory and molecular orbitals. Application of Sachs theorem. Theor. Chim. Acta. Berlin 26, 67–78 (1972) 75. Cvetkovic, D., Gutman, I., Trinajstic, N.: Graph theory and molecular orbitals. Croat. Chem. Acta. 44, 365–374 (1972) 76. Gutman, I., Trinajstic, N.: Graph theory and molecular orbitals. Total -electron energy of alternant hydrocarbons. Chem. Phys. Lett. 17, 535–538 (1972) 77. Gutman, I., Trinajstic, N.: Graph theory and molecular orbitals. IV. Further application of Sachs formula. Croat. Chem. Acta. 45, 423–429 (1973) 78. Gutman, I., Trinajstic, N.: Graph theory and molecular orbitals. V. Loop rule. Chem. Phys. Lett. 20, 257–260 (1973) 79. Gutman, I., Trinajstic, N.: Graph theory and molecular orbitals. Forschritte Chem Forschung (Topics in current chemistry) 42, 49–93 (1973) 80. Gutman, I., Trinajstic, N., Zivkovic, T.: Graph theory and molecular orbitals. VI. Discussion of nonalternant hydrocarbons. Tetrahedron 29, 3449–3454 (1973) 81. Gutman, I.: On the number of antibonding MO’s in conjugated hydrocarbons. Chem. Phys. Lett. 26, 85–88 (1974) 82. Trinajstic, N.: Chemical Graph Theory, vol. 1. CRC, Boca Raton, Florida (1983) 83. Randic, M.: Aromaticity and conjugation. J. Am. Chem. Soc. 99, 444–450 (1977) 84. Randic, M.: A graph theoretical approach to conjugation and resonance energies of hydrocarbons. Tetrahedron 33, 1905–1920 (1977) 85. Klein, D.J., Trinajstic, N.: Foundations of conjugated-circuits models. Pure. Appl. Chem. 61, 2107–2115 (1989) 86. Randic, M.: Aromaticity of polycyclic conjugated hydrocarbons. Chem. Rev. 103, 3449–3605 (2003) 87. Balaban, A.T., Randic, M.: Partitioning of -electrons in rings of polycyclic conjugated hydrocarbons. 5. Nonalternant compounds. J. Chem. Inf. Comput. Sci. 44, 1701–1707 (2004) 88. Jahn, H.A., Teller, E.: Stability of polyatomic molecules in degenerate electronic states. I. Orbital degeneracy. In: Proceedings of the Royal Society of London, Series A: Mathematical, Physical and Engineering Sciences, 161 220–235 (1937)

Chapter 7

The Central Role of Information Theory in Ecology Robert E. Ulanowicz

Abstract Information theory (IT) is predicated upon that which largely eludes physics – the absence of something. The capacity for IT to portray both presence and absence in comparable quantitative fashion makes it indispensable to ecology. IT has been applied to ecology along two separate lines: (1) it has been used to quantify the distribution of stocks and numbers of organisms and (2) it has been used to quantify the pattern of interactions of trophic processes. By and large, the first endeavor has resulted in relatively few insights into ecosystem dynamics and has generated much ambiguity and disappointment, so that most ecologists remain highly skeptical about the advisability of applying IT to ecology. By contrast, the second (and less wellknown) application has shed light on the possibility that ecosystem behavior is the most palpable example of a purely natural “infodynamics” that transcends classical dynamics, but remains well within the realm of the quantifiable. Keywords Ecological networks • Feedback control • Information measures • Robust stabilization • Sensitivity analysis • Sustainability • Theory of organization MSC2000 Primary 92B99; Secondary 94A17, 90B70, 93B52, 93D21, 49Q12.

1 A Fuller Picture of Nature The late Gregory Bateson [Bat72] divided the stuff of the natural world into two types – “pleroma,” by which he meant generic categories, such as energy, mass, or collections of homogeneous tokens, such as hydrogen gas; and “creatura,” the tokens

R.E. Ulanowicz () Department of Biology, University of Florida, Gainesville, FL 32611-8525, USA University of Maryland Center for Environmental Science, Solomons, MD 20688-0038, USA e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 7, © Springer Science+Business Media, LLC 2011

153

154

R.E. Ulanowicz

of which could be distinguished from one another by a “difference that makes a difference.” Pleroma was the stuff of conventional physics. In fact, Walter Elsasser [Els81] argued on logical grounds that the laws of physics can be formulated only under the conditions of homogeneity that defined Bateson’s pleroma. Furthermore, pleroma and the physics built upon them treat only that which exists in a positivist sense. Precepts in physics that invoke nonexistence are quite rare (e.g., the Pauli Exclusion Principle or Heisenberg’s Uncertainty). That “difference that makes a difference” Bateson called “information.” There are at least two ways in which Bateson’s perspective on information differs from the conventional. First, Bateson saw information as something that is constitutional or inherent in the entity that possesses it. This outlook differs substantially from the conventional rubrics of information theory, which are preoccupied with the semantics of communication. Furthermore, Bateson emphasized how differences between entities usually involve what is missing from an item. That is, information theory is built upon what eludes classical physics – namely, that which is absent [UGL09, Dea06]. If one accepts Bateson’s framework, it follows that classical physics is poorly suited to encompass a great deal of what transpires in biology, because the narrative of biology relies heavily upon patterns, and the majority of patterns in nature are juxtapositions of presences and absences. Any attempt to reduce biology entirely to physics is thus seen to be doomed from the outset. Furthermore, information theory, rather than being communications theory applied analogously to biology, is seen instead through Bateson’s lens to be central to and absolutely prerequisite for a full understanding of what biological systems are and how they behave. The constitutional necessity of information to biology is gradually being accepted by ever-more investigators as they begin to see beyond the simile of communications. It must be emphasized, however, that such ontological nature of information always focuses upon the relations. The very definition of ecology is predicated upon the relation, and this can be seen most explicitly in the subfield of ecosystem theory [Odu59]. Practitioners of ecosystem theory usually pay more attention to elucidating the relationships among the various populations that make up an ecosystem than they do to determining how much or what kind of organisms comprise those elements. Although some investigators prefer to think of such relations as analogous to communication, doing so is not necessary to invoke the power of quantitative information theory (IT). Over the history of ecology, information theory has been applied along two identifiably separate lines, which have met with disparate degrees of success: The application of information indices to quasi-static stocks or numbers has given rise to a number of problematic interpretations that at one point resulted in an almost wholesale repudiation of any role for IT in ecology [Ula01]. Because the majority of ecologists traffic in stocks and sizes, this failure has been widely advertised and has led many ecologists to eschew information theory. The echoes of this debacle persist to this very day. On the other hand, the use of information theory by a minority to quantify patterns of ecological processes has resulted in a wellspring of new insights and theories for ecology [JFB07]. Unfortunately, processes and flows are

7 The Central Role of Information Theory in Ecology

155

far more difficult to conceptualize and measure, and far fewer ecologists concern themselves with systems processes, so that the contributions of information theory to ecosystems research remain today relatively obscure.

2 A Propitious Beginning IT in ecology got off to a propitious start. Not long after Shannon [Sha48] formulated the beginnings of IT, Robert MacArthur [Mac55] interpreted how the new theory might be used to characterize networks of ecological processes or flows. Ecological networks provide a graphical picture of who eats whom within the community and at what rates each transfer of material or energy is occurring. His formulation drew heavily upon the earlier narratives of Raymond Lindeman [Lin42] and Eugene Odum [Odu53]. MacArthur followed Lindeman’s lead by describing the ecosystem as a configuration of quantified flows. MacArthur compared various configurations in the effort to identify which ones were likely to be most stable over time, and he drew his criterion for stability from E.P. Odum [ibid], “The amount of choice which the energy has in following the paths up through the food web is a measure of the stability of the community.” He argued that if one species happened to be abnormally common in the system, it would exert a small effect upon the rest of the community if its energy were distributed among a large number of predators. Similarly, if a population were abnormally uncommon, it could persist only if its predators had a large number of alternative hosts from which to draw their sustenance. MacArthur proposed the Shannon–Weaver index to gauge the uncertainty (choice) in the flows. If, for example, fi represents the i th flow, and F represents P the sum of all the flows .F D fi /, then pi D fi =F becomes the fraction that i

the i th flow comprises of the total system activity. MacArthur’s candidate for the measure of stability, S , then became: S D

X

pi log.pi /:

(7.1)

i

It is noteworthy that MacArthur made no effort to force his formalisms to comply with the categories of communication theory. MacArthur demonstrated extraordinary insight when he discussed the limitations of the index he had just proposed. He noted that stability rests upon species that have diets with many degrees of freedom; a restricted diet tends to lower stability. But a restricted diet is essential for efficiency, and both efficiency and stability are necessary for survival under natural conditions. That is, efficiency and stability tend to be antagonistic, but simultaneously necessary attributes of persistent systems. It therefore seemed at the time to MacArthur that (7.1) did not encompass all

156

R.E. Ulanowicz

the attributes necessary for survival. With those pertinent observations, MacArthur withdrew from the discourse, and his very promising lead remained virtually untouched for another 20 years.

3 A Costly Diversion Although many subsequent investigators acknowledged MacArthur’s 1955 paper as the origin for information theory in ecology, subsequent authors seemed not to have followed MacArthur’s lead very closely. Given the relative difficulty of measuring ecosystem flows, it was perhaps understandable that investigators might want to attempt alternative ways to measure system diversity. After all, there was precedent for measuring community diversity in terms of population sizes and stocks using conventional algebra [Sim49]. Hence, it was that investigators after MacArthur followed his lead by applying the Shannon–Weaver formula, not to system flows, but to the contents of each population. Margalef [Mar57, Mar61], for example, expressed pi not as the fraction of total activity, but rather as the fraction that a population comprises of the total number of organisms, or their total biomass. In symbols, he P set pi D ni =N , where ni is the number of organisms in population i and N D ni , so that the measure of biodiversity, D, became: i

DD

X

pi log.pi /:

(7.2)

i

For a number of reasons, not entirely unrelated to those cited by MacArthur, a higher biodiversity was thought to impart greater stability to the system. The ensuing argument that “diversity begets stability” provided ecology with a seemingly testable hypothesis, and the decade of the 1960s saw widespread and intense efforts among ecologists to test and elaborate the hypothesis [WS69]. Throughout the campaign, the Shannon index remained prominent among the candidates for quantifying biodiversity. Ways of quantifying stability, however, developed more slowly. Because dynamical stability analysis is a favorite tool of physicists, who are much admired by ecologists, that tool was readily accepted by ecosystems investigators when it was introduced by the erstwhile physicist turned ecologist, Robert May [May73]. May recapitulated the results of Gardner and Ashby [GA70], who studied the stability properties of randomly assembled collections of coupled linear differential equations. The latter investigators discovered that systems of low connectivity and moderate interaction tended to be inherently stable. As both the degree of connectivity and the strength of interaction increased, however, the systems became unstable. May used the stability criterion of Wigner [Wig59] to quantify the tipping point at which the probability for stability suddenly becomes negligible. The implication was that, for linear dynamics among randomly connected systems at least, diversity degrades stability.


157

May did caution that, since many diverse stable systems do exist, one should make the effort to discover why such diversity persists in ecosystems, but his warning went largely ignored. In the minds of most ecologists, it was May’s embarrassing refutation of “diversity begets stability” that persisted. Because May had stated his argument so eloquently, both mathematically and rhetorically, the feverish activity that had been devoted to the diversity-stability hypothesis collapsed overnight, and a widespread but tacit sense of humiliation swept through ecology. Information theory, because it had figured so prominently in the debate, became so tarred with failure in the minds of many ecologists that even today an undercurrent of this negative attitude persists – some 30 years after these events. IT remained ignored by ecologists, despite numerous opinions and evidence to the contrary [VOE80, Has84, TD94]. Despite the disdain for information theory as a theoretical tool for ecology, and the lack of any significant theoretical justification to support the maintenance of biodiversity, popular sentiment to conserve it continues strong into the present. Measuring the Shannon index of biodiversity remains common practice among those seeking to assess the integrity of an ecosystem. Due to its widespread popularity, the Shannon index has been studied in much detail – usually centering on issues such as the dual meaning of the term (number and evenness), or the statistics of how to estimate the index using finite collections [Pie69]. In fact, biodiversity has become such a prominent theme in environmental management that one is left to wonder why the fundamental assumption to focus on the diversity of population contents has received so little scrutiny?

4 MacArthur Revisited MacArthur had been frustrated by the ostensible inability of the Shannon measure of choice in energy flow to embody simultaneously both the notion of stability and that of efficiency. Demonstration that in fact both attributes do contribute to the Shannon measure was yet to come. The parsing out of contributions to complexity made by correlative constraints from those made by stochastic factors required that one consider conditionalities. Beyond regarding just the single distribution of probabilities, A, one must as well consider the distribution of a second character, B, and study how A distributes when it interacts with B. In short, one must consider relations. If we call p.ai / the probability that ai occurs; p.ai ; bj / the joint probability that ai and bj occur together; and p.ai jbj / the conditional probability that ai occurs, given that bj has transpired, then we may define a term that captures the relational constraints called the average mutual information, I.AI B/: X p.ai jbj / I i; j D 1; 2; 3; : : : ; n: (7.3) I.AI B/ D p.ai ; bj / log p.bj / i;j One can prove analytically that H.A/ I.AI B/ 0, and the interpretation of I.AIB/ is that it gauges how much of the complexity measured by H.A/ is

158

R.E. Ulanowicz

resolved by the action of B. That is, it measures the degree to which A is constrained by B (and vice versa, because I.AI B/ D I.BI A/ [the information is mutual]). The complement of I.AI B/ with respect to H.A/ is called the conditional entropy1 , H.AjB/ D H.A/–I.AI B/. H.AjB/ is likewise nonnegative. In effect, the Shannon index H.A/ has been decomposed into two components according to its relationship to B: H.A/ D I.AI B/ C H.AjB/; (7.4) where I.AI B/ describes how much A is correlated with (constrained by) B, and H.AjB/ quantifies how much freedom A exhibits in the presence of B. Rutledge et al. [RBM76] used this decomposition in a very clever way to resolve MacArthur’s predicament. They identified ai with the set of events, “a quantum of material or energy enters compartment i ”, while bj was denoted as “a quantum leaves compartment j .” Such identification makes all the measures reflexive (a key attribute of self-organizing systems). The interpretation given to (7.4) thereby becomes, “The complexity of the behavior of feeding (inputs) in the ecosystem equals the sum of the constraints that exist between predators and prey and the degree of choice left to the typical predator.” Since predator–prey constraints are usually associated with feeding efficiencies, it became explicitly clear how Shannon’s index embodies both elements of efficiency and freedom. Rutledge et al. suggested that H.AjB/ was the appropriate measure of choice extant in the system, and so should be related to system stability. Rutledge et al. also suggested how the three measures could be estimated in a way that was consistent with MacArthur’s earlier conventions. If Tij represents P the magnitude of flow from i to j , then the total system activity becomes T D Tij . i;j P The joint probability p.ai ; bj / takes the form Tij =T I p.ai / becomes Tij =T I p.bj / P j P Tij =T ; and p.ai jbj / is estimated by Tij = Tij . Using these is denoted by i

i

estimates in place of their corresponding probabilities yields the indices: 0P 1 0P 1 T Tij X B j ij C B j C H.A/ D A log @ A; @ T T i and

0 I.AI B/ D

X Tij i;j

T

(7.5)

1

Tij T B C P A; log @ P Tpj Tiq p

(7.6)

q

and H.AjB/ can be calculated by difference using (7.4).

1

The term entropy, chosen by Shannon, is retained only because of the ubiquity of its use. No relationship to thermodynamical entropy is implied thereby.


159

Rutledge et al. sought to quantify Odum’s idea that those systems with the most choice were the most stable and persisted longest. Thus, they suggested that the conditional entropy of flows could serve as the measure of system stability. Ulanowicz [Ula80], however, felt (like MacArthur) that system status should not be identified wholly with stability. True enough, when novel perturbations are significant, stability will play the paramount role in identifying system status. In the absence of major perturbations, however, it would seem that systems develop more along the lines of increasing efficiencies. He focused, therefore, upon mutual information as a measure of system organization. Earlier, in a seminal paper Eugene Odum [Odu69] had enumerated 24 attributes that could be used to gauge ecosystems maturity. Ulanowicz noted significant correspondence between many of Odum’s indices and the average mutual information. Howard Odum [OP55], however, had shown that efficiency is a poor indicator of system development during its inchoate early stages. During the incipient stages of development, often when resources are abundant, system changes seem to be characterized more by increases in power throughput. In an effort to encompass the whole course of system development into a single index, Ulanowicz borrowed from Myron Tribus [TM71] and scaled the average mutual information of the network of trophic processes (organization) by the total system throughput, T , (power) to create a new variable, which he called the system ascendency, A. That is: A D T I.AI B/ 0 1 X Tij T B C P A: D Tij log @ P T T pj iq i;j p

(7.7)

q

Because many of the Odum’s criteria for development appeared to correlate with increases in the ascendency index, Ulanowicz proposed as a phenomenological principle: In the absence of major perturbations ecosystems exhibit a propensity to increase in ascendency. It is always the case, however, that perturbations do intervene; so it was necessary also to track the impacts of such interventions. Now, Rutledge et al. had used the conditional entropy H.A/–I.AI B/ to measure the freedom or flexibility of the system to respond to such perturbations. Ulanowicz and Norden [UN90], however, pointed out that this difference is biased in favor of inputs over outputs, and proposed that the upper limit for the ascendency should be estimated by the joint entropy instead. That is, they scaled the difference between the joint entropy and the mutual information by the total system throughput (as done earlier with the ascendency) to yield a more symmetrical system property, ˚, called the system overhead. ˚ D H.A; B/ I.AI B/ 0 1 2 X Tij Tij B C P A: D log @ P T T T pj iq i;j p

q

(7.8)

160

R.E. Ulanowicz

As with Rutledge et al., they inferred that the stability of the system is maintained by the disorganized, incoherent, inefficient system processes that contribute to this overhead. Under quiescent or benign conditions, these inefficiencies tend to hamper efficient system performance. During times of novel or stochastic stress, however, the same collection of such processes acts as a reservoir from which the system can draw to reconfigure itself (adapt) to the perturbation. It appears that some degree of each of the mutually exclusive attributes of organization and flexibility are required if a system is to remain sustainable over the long run [Ula00a]. It seems that MacArthur was prescient in suggesting that system integrity or sustainability should be related to the sum of these agonistic measures. In quantitative terms, the system capacity, C , can be gauged by the product of the total system throughput and the joint entropy: C D T H.A; B/ D AC˚ X Tij : D Tij log T i;j

(7.9)

Because the capacity serves as an upper bound on both A and ˚, anything which would contribute to an increase in C would also provide capacity for either ascendency or overhead to grow. Brooks and Wiley [BW86], for example, argued that the natural tendency for C to grow is analogous to the thermodynamic drive toward increasing the entropy of the universe. Critics of the information theoretic approach often point out that the suite of variables defined thus far can be applied only to static or quasi-static configurations of flows. Real ecosystems, to the contrary, are heterogeneous in space and dynamical in time. Although gathering sufficient data to quantify networks over time and space can at times be an overwhelming task, no conceptual barrier exists to extending these indices over the temporal and spatial intervals. All the information indices defined above can be extended into three or more dimensions [Abr63]. Claudia Pahl-Wostl [Pah92] cautions, however, that the proper extension for the average mutual information in three dimensions is not the three-dimensional counterpart, I.AI BI C /, but rather the cluster fI.AI BI C / C I.AI BjC / C I.BI C jA/ C I.C I AjB/g. In quantitative terms, if Tijk denotes the amount of medium flowing from i to j during time interval k (or in spatial cell k), then the corresponding three-dimensional ascendency becomes: 0 AD

X i;j;k

1

B Tijk log @ P p

2 Tijk T

Tpjk

P q

Tiqk

P r

Tijr

C A:

(7.10)

In a similar manner, four and higher dimensional versions of ascendency can be constructed to treat even more complicated systems. The principle of increasing


161

ascendency is presumed to apply as well to spatial and temporal dynamics. That is, systems are assumed to adapt to perturbations that possess regularities in time and/or space by adjusting their flow distributions in (7.10) so as to achieve higher values of the system ascendency. Thus, the principle of increasing ascendency answers, in part at least, one of the central questions in ecology, “Why are organisms distributed over time and space according to observed patterns?” Explaining such patterns should be facilitated by the application of the principle of increasing (multidimensional) ascendency [Ula00b]. (One recalls from the introduction how pattern involves the quantification of absences, and is thus naturally accommodated by IT.) Yet another criticism of the application of IT to this point could be that all measures are estimated solely in terms of processes, and dynamics are not controlled by the rates of processes alone. True, the earlier preoccupation in ecology with population numbers and stocks may have been misleading, but it remains equally inconceivable that such attributes should play no role in guiding system dynamics. How then to introduce compartmental stocks into the calculation of the ascendency in a manner that is consonant with the formalisms of IT? One approach [UA97] might be to regard the a priori probability for an exchange of medium between i and j to be proportional to the product of the stocks contained in those respective nodes (the law of mass action.) Thus, if Bi represents the level of the contents of i and Bj that of j , then the a priori joint probability that medium P will flow from i to j could be estimated by the product Bi Bj =B 2 , where B D Bi . The corresponding i

a posteriori probability could be taken to be the fraction that the observed flow, Tij , constitutes of the total activity, i.e., Tij =T . The weighted difference between the a posteriori and a priori probabilities is what in IT is called the Kullback–Leibler difference. In terms of the measured quantities already described, the biomassinclusive form of the ascendency, Ab , becomes: Ab D

X Tij i;j

T

log

Tij B 2 ; Bi Bj T

(7.11)

which can be generalized for three or more dimensions as discussed in the previous paragraph. The partial derivatives of the ascendency with respect to either flows, Tij or biomasses, Bi represent the sensitivities of the whole system status with respect to changes in those individual system elements. Typically, large values of these derivatives signify where resistance is controlling further system development. Ulanowicz and Baird [UB99, SCO99] therefore used a sensitivity analysis of the spatial networks of several nutrient elements in the Chesapeake Bay ecosystem to pinpoint which element should control system dynamics during each season. Significantly, they discovered instances when the largest flow into or out of a system compartment was not the most controlling link. Control in dynamical networks does not always reside in the links of greatest activities.

162

R.E. Ulanowicz

5 Infodynamics MacArthur paid little heed to the conventions of communications theory as he framed his index of flow diversity, and the subsequent elaborations discussed above have departed even further afield (unlike the applications of IT to molecular biology, wherein coding and communication are explicitly assumed). Instead, ascendency is taken to represent the overall effect of numerous unidentified constraints at various scales that serve to guide physical medium through the ecosystem. Collier [Col90] called such constraints “enformation” to distinguish their constitutional nature from that of conventional information. In one sense, at least, information theory applied to ecosystem networks makes explicit and measurable the effects of “signs” discussed elsewhere under the rubric of “biosemiotics” [Hof93, Hof08]. More than the constitutive nature of information is at issue, however. System “dynamics” can take on a radically new form once information and pattern can be quantified. In conventional science, the world is perceived as a unidirectional hierarchy, wherein causes arising at microscopic scales ramify upward to generate macroscopic patterns. The goal of theory usually is to describe events at the lowest level possible in terms of deterministic mechanical dynamics. The effects of these mechanical agencies may be modified by chance interferences, but their effect is believed to be expressed at higher levels in some simple aggregated fashion. In the introduction, we discussed another possibility – that the dynamics expressed in terms of information variables are both legitimate and necessary for a full understanding of phenomena. That is, the world is not structured simply by the machinations of energy and mass. Structures, such as those exhibiting positive feedback or autocatalysis, can act as agencies in the sense of Aristotle’s formal cause [Ula99], and their effects can ramify downward as well as up the hierarchy of scales. In this view, the increase in ascendency is more than a phenomenological quirk to be explained away by reductionistic exegesis. Increasing ascendency constitutes the controlling dynamic. Particular mechanisms and elements may come and go and are selected for (and against) according to the driving dynamic. A scenario wherein information plays a key causal role has been termed “infodynamics” by Weber et al. [WDD89], and the suggestion here is that ecology may be governed as much (or more) by infodynamics as by the conventional sort. In response to the last assertion, some readers may ask just where in the indices that have been defined do the dynamic lie? One response might be that increasing ascendency represents changing constraints per se, but that is only part of the story. A peculiar formal property of the ascendency is that, even when it is calculated on a static network, clues to the dynamics inherent in the network pattern are built into the index itself. To appreciate this serendipitous circumstance, one need to only calculate the derivatives of the ascendency with respect to the individual flows. Thus, taking the derivative of (7.7) with respect to Twx , one arrives, after considerable algebra, at the result:


163

0

1

Twx T @A B C P D log @ P A; @Twx Tpx Twq p

(7.12)

q

so that the equation for the ascendency can be rewritten as: AD

X i;j

Tij

@A : @Tij

(7.13)

But this relation is homologous to the chain rule: dA D

X @A dTij : @Tij i;j

(7.14)

One may conclude then that whenever information theory is applied to the system structure, the measures provide implicit clues about how the system is likely to change. This contrasts with the situation in conventional dynamics, where information on static configurations tells one nothing at all about the system’s dynamical behavior.

6 Ascendency, Capacity, and Sustainability The relationships between capacity, ascendency, and overhead make it an easy task to quantify the degree of order or constraint inherent in a quantified network. Because C A 0, the ratio A=C will always occupy the closed interval [0, 1]. Defining a D A=C , it becomes appropriate to identify a with the “degree of organization” inherent in a flow network. When a D 0, there either is no connection whatsoever among the units under study (as in a perfect gas), or else every component is connected with every other in equiponderant fashion. At the other extreme (in the limit as a!1), the system approaches a closed cycle of equiponderant flows having no connections to the outside. Both extremes are known in thermodynamics as equilibrium systems. Systems falling between these endpoints will be an admixture of constraint and incoherence. They are open systems that are not at thermodynamic equilibrium. The question arises as to how are natural ecosystems distributed over the range of a? A widespread intuition is that systems become more ordered as they develop [Odu69], so that one would expect a to be lower in immature systems and higher in mature ones (as suggested following (7.3) above). It turns out that the range of a is narrower than what might be expected under such assumption, falling within what has been called the “window of vitality” [Ula02, ZU03]. Ricard Solé has independently observed the same clustering of values [SV04]. Furthermore, if consideration is limited to networks that are quantified in detail (say, having

164 1 0.8 Robustness

Fig. 7.1 The distribution of real ecosystem flow networks of moderate complexity .n > 12/ with respect to degree of order .a/ and robustness .a logŒa/. Data from Zorach and Ulanowicz [ZU03]

R.E. Ulanowicz

0.6 0.4 0.2 0 0

0.2

0.4 0.6 Degree of Order

0.8

1

more than about 12 components), the clustering of a becomes tighter still (Robert Christian, personal communication), ranging from about 0.32 to 0.48. In order to investigate the reasons behind the narrow range in a, we return to the fundamental notions of being and nonbeing that provided the foundations for IT (as discussed at the beginning of this essay). Then, if a characterizes the degree of order or constraint within a system, one could identify, in analogy with Boltzmann [Bol72] and Gibbs, Œ log.a/ as an appropriate measure of system flexibility (as opposed to Œ1 a, which straightforward probability would suggest). The product Œa log.a/ would then represent the combination of order and flexibility that a robust, sustainable system would possess. Figure 7.1 depicts how real, moderately complex ecosystems plot with respect to a and Œa log.a/. It is immediately evident that natural ecosystems cluster near maximal robustness. At present, one can provide only generic reasons behind the clustering phenomenon. Systems to the left of the maximum of Œa log.a/, which occurs at a D .1=e/, can acquire further robustness via autocatalytic feedbacks, which serve to increase a [Ula97]. Too much ascendency can become a liability, however, whenever the system passes well to the right of maximal robustness. The erosion of overhead in the wake of increasing ascendency will render the system vulnerable to chance perturbations, and eventually it will collapse in an “avalanche” to the left. Cyclical patterns of growth and collapse characterize some ecosystems [Hol86] and appear endemic to business cycles in the global economy. Thus, how to manage systems to remain in proximity to maximal sustainability becomes a vital issue of global importance [GLU09].

7 Summary and Conclusions It appears that the discipline of ecology will continue to be pursued along two separate lines. The majority of ecologists will continue in their studies of single populations in relation to an external environment. Emphasis in these investigations


165

is upon population size and contents – i.e., exclusively upon that which is. Here relationships of the object species with other populations remain of secondary concern. Because IT arises out of a relational context and provides a measure for that which is missing in a pattern, it is an awkward fit with conventional studies, and the majority will continue to hold IT in low esteem. Systems ecology, by contrast, focuses primary attention upon the relationships of populations with one another. It is concerned with the pattern of those relationships, in which the absence of links plays a role comparable to that of their presence. The convenient fit of IT to these endeavors has generated a number of enticing hypotheses and fruitful phenomenological observations. However, because system ecology remains such a minority endeavor, it is questionable whether such promise will ever be embraced by the larger community of ecologists. The promise of IT can perhaps be illustrated by how IT can serve the popular issue of the conservation of biodiversity. There seems to exist an overwhelming consensus that global biodiversity is worth preserving. Earlier use of IT to quantify biodiversity in terms of populations has failed to justify such preservation, preoccupied as conventional theory is with contents and mechanical dynamics. Information theory used to quantify the diversity of trophic flows (as MacArthur had originally suggested) opens, on the other hand, a whole new vista on relational dynamics – one which considers both what is present and what is not and provides ready justification for protecting the diversity of ecosystems processes. Perhaps, it is time to reconsider the assumptions we use to view the living world and to advance on the promise evident in a shift away from conventional mechanical dynamics and toward the development of a robust infodynamics.

References [Abr63] [Bat72] [Bol72] [BW86] [Col90]

[Dea06]

[Els81] [GA70] [GLU09]

Abramson, N.: Information Theory and Coding. McGraw-Hill, New York (1963) Bateson, G.: Steps to an Ecology of Mind. Ballantine Books, New York (1972) Boltzmann, L.: Weitere Studien ueber das Waermegleichtgewicht unter Gasmolekuelen. Wien. Ber. 66, 275–370 (1872) Brooks, D.R., Wiley, E.O.: Evolution as Entropy: Toward a Unified Theory of Biology. University Chicago Press, Chicago (1986) Collier, J.D.: Intrinsic information. In: Hanson, P.P. (ed.) Information, Language and Cognition: Vancouver Studies in Cognitive Science, Vol. 1, pp. 390–409. University of British Columbia Press, Vancouver (1990) Deacon, T.W.: Emergence: The hole at the wheel’s hub. In: Clayton, P., Davies, P. (eds.) The Re-emergence of Emergence: The Emergentist Hypothesis, pp. 111–149. Oxford University Press, London (2006) Elsasser, W.M.: A form of logic suited for biology. In: Robert Rosen (ed.) Progress in Theoretical Biology, pp. 23–62. Academic Press, New York (1981) Gardner, M.R., Ashby, W.R.: Connectance of large dynamic (cybernetic) systems: critical values for stability. Nature 228, 784–784 (1970) Goerner, S.J., Lietaer, B., Ulanowicz, R.E., Gomez, R.: Quantifying economic sustainability: implications for free-enterprise theory, policy and practice. Ecol. Econ. 69, 76–81 (2009)

166 [Has84] [Hof93] [Hof08] [Hol86]

[JFB07]

[Lin42] [Mac55] [Mar57] [Mar61] [May73] [Odu53] [Odu59] [Odu69] [OP55]

[Pah92] [Pie69] [RBM76] [SCO99] [Sha48] [Sim49] [SV04] [TD94] [TM71] [Ula80] [Ula97] [Ula99] [Ula00a]

[Ula00b]

R.E. Ulanowicz Hastings, H.M.: Stability of large systems. BioSystems 17, 171–177 (1984) Hoffmeyer, J.: Signs of Meaning in the Universe. Indiana University Press, Bloomington, Indiana (1993) Hoffmeyer, J.: Biosemiotics: Signs of Life and Life of Signs. University of Scranton Press, Scranton, Pennsylvannia (2008) Holling, C.S.: The resilience of terrestrial ecosystems: local surprise and global change. In: Clark, W.C., Munn, R.E. (eds.) Sustainable Development of the Biosphere, pp. 292–317. Cambridge University Press, Cambridge, UK (1986) Jørgensen, S.E., Fath, B.D., Bastianoni, S., Marques, J., Mueller, F., Nors-Nielsen, S., Patten, B.C., Tiezzi, E., Ulanowicz, R.E.: A New Ecology: Systems Perspective. Elsevier, Amsterdam (2007) Lindeman, R.L.: The trophic-dynamic aspect of ecology. Ecology 23, 399–418 (1942) MacArthur, R.: Fluctuations of animal populations, and a measure of community stability. Ecology 36, 533–536 (1955) Margalef, R.: La teoria de la informacion en ecologia. Mem. Real Acad. Ciencias Artes Barcelona 32(13), 373–449 (1957) Margalef, R.: Communication of structure in planktonic populations. Limnol. Oceanog. 6, 124–128 (1961) May, R.M.: Stability and Complexity in Model Ecosystems, p. 235. Princeton University Press, Princeton, NJ (1973) Odum, E.P.: Fundamentals of Ecology. Saunders, Philadelphia (1953) Odum, E.P.: Fundamentals of Ecology, 2nd edn. Sanders, Philadelphia, PA (1959) Odum, E.P.: The strategy of ecosystem development. Science 164, 262–270 (1969) Odum, H.T., Pinkerton, R.C.: Time’s speed regulator: the optimum efficiency for maximum power output in physical and biological systems. Am. Sci. 43, 331–343 (1955) Pahl-Wostl, C.: Information theoretical analysis of functional temporal and spatial organization in flow networks. Math. Comput. Modell. 16(3), 35–52 (1992) Pielou. E.C.: An Introduction to Mathematical Ecology. Wiley-Interscience, New York (1969) Rutledge, R.W., Basorre, B.L., Mulholland, R.J.: Ecological stability: an information theory viewpoint. J. Theor. Biol. 57, 355–371 (1976) SCOPE: Ecosystem ascendancy and nutrient dynamics. SCOPE Newslett. 93, 4–5 (1999) Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948) Simpson, E.H.: Measurement of diversity. Nature 163, 688 (1949) Solé, R.V., Valverde, S.: Information theory of complex networks: On evolution and architectural constraints. Lect. Notes Phys. 650, 189–207 (2004) Tilman, D., Downing, J.A.: Biodiversity and stability in grasslands. Nature 367, 363–365 (1994) Tribus, M., McIrvine, E.C.: Energy and information. Sci. Am. 225, 179–188 (1971) Ulanowicz, R.E.: An hypothesis on the development of natural communities. J. Theor. Biol. 85, 223–245 (1980) Ulanowicz, R.E.: Ecology, the Ascendent Perspective. Columbia University Press, NY (1997) Ulanowicz, R.E.: Life after Newton: An ecological metaphysic. BioSystems 50, 127–142 (1999) Ulanowicz, R.E.: Toward the measurement of ecological integrity. In: Pimentel, D., Westra, L., Noss, R.F. (eds), Ecological Integrity: Integrating Environment, Conservation and Health, pp. 99–113. Island Press, Washington, DC (2000) Ulanowicz, R.E.: Quantifying constraints upon trophic and migratory transfers in spatially heterogeneous ecosystems. In: Sanderson, J., Harris, L.D. (eds.) Series in Landscape Ecology: A Top-Down Approach, pp. 113–142. Lewis Publications, Boca Raton, FL (2000)

7 The Central Role of Information Theory in Ecology [Ula01] [Ula02]

167

Ulanowicz, R.E.: Information theory in ecology. Comput. Chem. 25, 393–399 (2001) Ulanowicz, R.E.: The balance between adaptability and adaptation. BioSystems 64, 13–22 (2002) [UA97] Ulanowicz, R.E., Abarca-Arenas, L.G.: An informational synthesis of ecosystem structure and function. Ecol. Model. 95, 1–10 (1997) [UB99] Ulanowicz, R.E., Baird, D.: Nutrient controls on ecosystem dynamics: The Chesapeake mesohaline community. J. Mar. Sci. 19, 159–172 (1999) [UGL09] Ulanowicz, R.E., Goerner, S.J., Lietaer, B., Gomez, R.: Quantifying sustainability: resilience, efficiency and the return of information theory. Ecol. Complex. 6, 27–36 (2009) [UN90] Ulanowicz, R.E., Norden, J.: Symmetrical overhead in flow networks. Int. J. Syst. Sci. 21(2), 429–437 (1990) [VOE80] Van Voris, P., O’Neill, R.V., Emanuel, W.R., Shugart, H.H. Jr.: Functional complexity and ecosystem stability. Ecology 61, 1352–1360 (1980) [WDD89] Weber, B.H., Depew, D.J., Dyke, C., Salthe, S.N., Schneider, E.D., Ulanowicz, R.E., Wicken, J.S.: Evolution in thermodynamic perspective. Biol. Phil. 4, 373–405 (1989) [Wig59] Wigner, E.P.: Statistical properties of real symmetric matrices with many dimensions. In: Proceedings of the Fourth Canadian Mathematical Congress, Toronto 174 (1959) [WS69] Woodwell, G.M., Smith, H.H.: Diversity and Stability in Ecological Systems, p. 22. U.S. Brookhaven Symp. Biol., NY (1969) [ZU03] Zorach, A.C., Ulanowicz, R.E.: Quantifying the complexity of flow networks: How many roles are there? Complexity 8(3), 68–76 (2003)

Chapter 8

Inferences About Coupling from Ecological Surveillance Monitoring: Approaches Based on Nonlinear Dynamics and Information Theory L.J. Moniz, J.D. Nichols, J.M. Nichols, E.G. Cooch, and L.M. Pecora

“It is far better to foresee even without certainty than not to foresee at all.” - H. Poincare “Physics envy is the curse of biology” - J. Cohen

Abstract Some monitoring programs for ecological resources are developed as components of larger science or management programs and are thus guided by a priori hypotheses. More commonly, ecological monitoring programs are initiated for the purpose of surveillance with no a priori hypotheses in mind. No conceptual framework currently exists to guide the development of surveillance monitoring programs, resulting in substantial debate about program design. We view surveillance monitoring programs as providing information about system dynamics and focus on methods for extracting such information from time series of monitoring data. We briefly describe methods from the general field of nonlinear dynamics that we believe may be useful in extracting information about system dynamics. In looking at the system as a network of locations or components, we emphasize methods for assessing coupling between system components for use in understanding system dynamics and interactions and in detecting changes in system dynamics. More specifically, these methods hold promise for such ecological problems as identifying indicator species, developing informative spatial monitoring designs, detecting ecosystem change and damage, and investigating such topics as population synchrony, species interactions, and environmental drivers. We believe that these ideas and methods provide a useful conceptual framework for surveillance monitoring and can be used with model systems to draw inferences about the design of surveillance monitoring programs. In addition, some of the current methods should be useful with some actual ecological monitoring data, and methodological

L.J. Moniz () Johns Hopkins University, Applied Physics Laboratory, 11100 Johns Hopkins Road, Laurel, MD 20723, USA e-mail: [email protected]; [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 8, © Springer Science+Business Media, LLC 2011

169

170

L.J. Moniz et al.

extensions and modifications should increase the applicability of these approaches to additional sources of actual ecological data. Keywords 37M10 Time-Series Analysis • 37M99 Computational Methods • 37N99 Application of Dynamical Systems • 92D40 Ecology

1 Introduction The monitoring of ecological systems is an activity that is increasingly common throughout the world [YNB01] and, as such, is beginning to be accompanied by increased scrutiny, as scientists and natural resource managers seek to insure that scarce funds are expended wisely. Scrutiny has been focused on topics ranging from the reasons for monitoring, to the selection of system attributes to be monitored, to the manner in which selected system attributes are estimated. Indeed, Yoccoz et al. [YNB01] challenged those engaged in the conduct or development of monitoring programs to pose the three basic questions: why, what, and how? They emphasized that answers to the what and how questions are conditional on the answer to the question, why monitor? We begin by asserting that monitoring is not a stand-alone activity but is most usefully viewed as a component of a larger program of either science or management [Njd00, YNB01, NW06]. The role of monitoring in most scientific programs is to provide data on changes in system variables that can be used to confront predictions from models of competing hypotheses about system dynamics [HM97, WNC02, NW06]. This confrontation can lead to a rejection of hypotheses under some approaches to science and to a change in the likelihoods or degrees of faith associated with the different hypotheses under other approaches to science. The roles of monitoring in most management programs similarly include confrontation with predictions from models of competing hypotheses about system responses to management, as well as provision of estimates of system state for use in making state-dependent decisions, and estimates of state and related variables for use in judging management effectiveness. We view these uses of monitoring in the conduct of science and management as ideal, in the sense that monitoring results play a key role in discriminating among competing hypotheses and in making informed management decisions. However, we believe that many monitoring programs have not been developed to discriminate among a priori hypotheses about system behavior or to provide estimates of system state for the purpose of making state-dependent management decisions. We will refer to such monitoring that is not designed with reference to guiding hypotheses about system behavior or response to management as surveillance monitoring. We do not view surveillance monitoring as an efficient way to use conservation funds, yet a substantial number of such programs exist worldwide. Here we consider approaches to the analysis of such monitoring data that might be useful for trying to learn about a system and its dynamics.

8 Inferences About Coupling from Ecological Surveillance Monitoring: Approaches: : :

171

We contend that, in contrast to hypothesis-driven monitoring, no methodological framework has been presented to guide the design and analysis of surveillance monitoring data. We have noted that hypothesis-driven monitoring is used to conduct either science or management. In the case of science, there is a welldeveloped body of statistical theory about how to test hypotheses using collected data, both in general [Fis47, Fis58] and specifically in the case of ecological data [Hur84, SR92, SG93, WNC02]. In the case of management, a similarly welldeveloped body of decision theory is available to guide use of data to make optimal decisions in the face of uncertainty [Wil82, Wil89, Wil96, WNC02]. In contrast, there is a noted absence of true theory guiding the collection and use of data in ecological surveillance monitoring programs. However, investigators in other disciplines have developed theory that is relevant to investigative uses of such data, formally addressing such questions as: Are the measured data sufficient for us to understand the evolution of the dynamical system? In particular, do the measured data contain enough information to reconstruct dynamical objects of interest and recover coordinate independent dynamical properties? [OY03]. Our contention is that this work in other fields can potentially be useful in developing a general theory guiding both the collection of time series of ecological data and the use of such data in a manner that extracts maximal information about the underlying systems of interest. Although many of the methods used in the fields of physics and nonlinear dynamics were developed for long time series of relatively noise-free data, we believe that these methods have significant potential to: 1. Provide a framework for extracting information from ecological time series 2. Provide us with upper limits on the information extraction that is possible 3. Be used with data from model systems, perhaps leading to generalizations that may be useful, for example, in the design of monitoring programs 4. Be adapted or extended to deal with the sorts of short, noisy data that tend to characterize ecological time series [Bou96, Bou01, PSWM00] We consider surveillance monitoring programs that provide time series of one or more state variables of a system of interest. State variables might be abundances of different species (e.g., a network of interconnected species), or perhaps multiple observations of the same species from a network of different spatial locations within the system. The task at hand is then to consider the time series as sources of information about the system of which they are a part and consider the kinds of information that can be extracted as well as the methods needed to accomplish this extraction. We focus on methods developed for nonlinear systems, because of the apparent prevalence of nonlinearity in ecological systems and processes [CCDD95, CDCD97, CCDDH03, DDCC95, DDCC97, HHETG93, Sch81, SEK86, Tur03]. In particular, we focus on methods for assessing coupling (dynamical interdependence; [PCH97, SSCBS96]) of different potential system components or state variables, for purposes such as: 1. Identifying whether two variables are indeed components of the same system 2. Investigating the nature of interactions among system components

172

L.J. Moniz et al.

3. Drawing inferences about one or more system components by monitoring another component(s) 4. Investigating coupling within a network of spatial locations or interrelated species The methodological approaches to the investigation of coupling for nonlinear systems have been categorized as based on either a geometric portrait of the system dynamics or on the information content of dynamical system components. As Sauer [Sau04] noted about the geometric approach, “The capability of these methods for nonlinear systems, built on advanced reconstruction techniques, far exceeds the reach of conventional signal processing.” Our objective is simply to consider the use of these approaches to extract information about systems and their behaviors from time-series data that are collected in the absence of a priori hypotheses and corresponding models. We note that although we are applying the following techniques to the specific settings of ecological monitoring of a network of either interrelated species or a network of spatial locations, the techniques are suited to investigation of the same phenomena in a general network setting. Throughout this chapter, we view the terms network and system similarly – that is, as a group of entities or components that are not independent (e.g., abundances of multiple species at different locations) and that have linked or connected dynamics. We emphasize that the focus of this chapter is on the use of time-series data to draw inferences about ecological systems and networks. Use of these and other approaches to inference should result in the development of models of ecological systems. Such system models can then be analyzed using a variety of approaches (e.g., graph theory) for purposes such as decomposition and identification of compartments and subsystems [ABB06, CGC06] and identification of “important” species [BJP09, JOBL08, Jor09].

2 System Identification By system identification, we simply mean that we would like to use time series from one or more state variables from a system of interest to draw inferences about characteristics of the entire system. Ecological systems can contain large numbers of potential state variables (e.g., populations of various species at different locations in a spatially extended system), such that attempts to monitor all such variables would be extremely difficult and expensive, at best, and more typically impossible [MZSM04]. However, approaches from nonlinear dynamics offer hope to the ecologist of being able to learn something about the dynamics of an entire multidimensional system based on a time series of one or a few state variables. This possibility causes us to consider the use of these same approaches to identify those state variables that yield the greatest information about system dynamics.


173

A useful geometric description of dynamics for a multidimensional system is based on the concept of an attractor, which is a closed set of points in state space (e.g., defined by the abundance of each interacting species in the system). System trajectories beginning on the attractor remain on it, whereas trajectories beginning near (specifically, in the associated attracting set) the attractor will converge to it (for more precise definitions of attractor, see [Mil85, Str94]). The attractor is thus the portion of state-space in which the system tends to remain and to which it returns following perturbation. The geometry of an attractor thus contains a great deal of information about system dynamics. Indeed, ecological interest in a system’s attractor is analogous to interest in stable equilibria for systems of multiple competing species [May73]. The strict definition of an attractor assumes that the underlying dynamics are deterministic. However, the concept of state-space can be quite useful with or without such an assumption. A state-space view of system dynamics provides the practitioner with a picture of the relative frequencies (probabilities) of the system’s returning to a given state. Many of these geometric approaches are based on local probability density estimates (or closely related measures on an attractor; [ER95]) and/or the transition probabilities of moving from one location to another in the state space. This connection between geometric and probability density descriptions is important to the understanding of how attractorbased approaches to system identification can be applied to ecological systems. Thus, attractor-based methods may be appropriately viewed as a special subset of density-based approaches to time-series analysis. This view also blurs the distinction between approaches based on geometry versus information content. The concept of an attractor is typically applied to stationary systems, such that the rules governing system dynamics are not changing over time periods of interest. If systems themselves are changing over a period of interest, then the concept of an attractor may no longer be useful. However, note that such change presents problems, not only for attractor-based approaches, but also for virtually any approach to the conduct of science. Science is based on predictions, and if systems change so rapidly that there is no basis for prediction (historic system dynamics provide no information about future dynamics), then there is little hope for doing anything other than describing change. A second comment concerns the possibility of using geographic replication to draw inferences about system attractors. The assumption of stationarity is needed for single time series to provide temporal replication or multiple looks at system dynamics as the system travels through state space. If multiple independent looks at system dynamics are provided by geographic replication, then systems need not be stationary for long periods to draw inferences about their respective attractors. However, we note again that learning about system dynamics during one time period may not be informative about future times if substantive changes in dynamics occur. Later in this chapter, we focus on detecting such change. Finally, we note recent work indicating that the concept of an attractor still provides a useful description for systems that experience specific kinds of change, for example systems subjected to either deterministic or stochastic forcing [SBDH97, Sta99, SBDH03].

174

L.J. Moniz et al.

Construction of an attractor for a multispecies ecological system appears to be a daunting task, requiring time series of abundances for each species in the system. However, Takens embedding theorem ([Tak81], also see [PCFS87, SY91, Whi36, Yul27]) provides an approach by which the attractor of a multidimensional system can be qualitatively reconstructed (such that the reconstructed attractor is diffeomorphic – geometrically equivalent – with respect to the original attractor) from a time series of data from a single system state variable (e.g., a single species). Specifically, delay coordinates of the single series are used to construct new pseudo time series from the original data. Reconstruction requires selection of the number of delay coordinates to use (the embedding dimension) and the length(s) of the delay [PMNC07, Wil97]. This basic approach to attractor reconstruction has been used successfully for long deterministic time series of single state variables from systems that are not of very high dimension. If time-series data from more than one system state variable are available, then delayed copies from multiple time series can be used in the embedding, again presenting an interesting optimization problem that requires selection of appropriate state variables and their respective delay copies [PMNC07]. Ecological time series are typically short and noisy, leading to the obvious question of what to do in this situation. In some cases, addition of noise obscures underlying determinism, whereas in other cases attractor-based approaches still provide reasonable inference about deterministic structure [SEK86]. This result anticipated the demonstration by Casdagli [Cas92] that attractor reconstruction also applies to input–output systems with stochastic input. It is likely that reasonable models of state variables of ecological systems will include deterministic components as well as term(s) representing the actions of exogenous variables that are modeled as random noise. Despite the existence of such random noise terms, Cheng and Tong [CT92] note that the search for low dimensional attractors is a meaningful signal extraction exercise.

3 Coupling and Comparative Dynamics The preceding material involved efforts to draw inferences about an entire dynamical system based on the time series of one or more system state variable(s) [PMNC07]. In addition, we would frequently like to use two or more time series to address two general classes of problems. The first class involves investigation of interactions or coupling among state variables hypothesized to be components of the same system. Such investigations not only permit inference about the structure of ecological systems (e.g., nature and degree of interaction) but also address the ability to draw inferences about one system component by monitoring another. The second class of problem involves comparison of two time series, perhaps of the same system component, but from different locations or periods of time, to test for possible


175

differences in system dynamics. Such comparisons will be relevant to inferences about change in ecological systems, regardless of whether change is hypothesized to be associated with an identified perturbation.

3.1 Coupling If two state variables are components of the same system, then they exhibit dynamical interdependence [PCH97, SSCBS96]. The existence of dynamical interdependence underlies the ability to reconstruct a system’s attractor from a time series of a single state variable. Coupling between different biological state variables implies the existence of one of a number of kinds of interactions of substantial interest to ecologists. If different species are involved, then trophic (e.g., predator–prey, food web) or competitive interactions are suggested. For populations of the same species at different locations, active dispersal can result in dynamical interdependence. Linear cross-correlation has been used frequently by ecologists to investigate relationships between state variables, especially single species populations at different locations [BIL99, Koe99, KR02, PF02, RKL98]. Linear cross-correlation assumes a very specific functional relationship that is symmetric in its argument and defines coupling in terms of second-order correlations only, i.e., EŒx.t/; y.t C T /, where x.t/; y.t/ are the values of two state variables at time t. Here we consider methods that are based on the entire probability density structure (i.e., correlations of any order) and are therefore more generally applicable and capable of dealing with nonlinear systems. The methods we consider also lead to inferences about asymmetries between system components. These asymmetries can involve information and/or dynamical influence and are of great potential interest to ecologists [Njm05, NMNPC05].

Attractor-Based Approaches Some methods for investigation of coupling are based on reconstructed attractors for each of the two state variables of interest. Continuity statistics are based on the fact that if two state variables belong to the same system, then their respective attractors must be related by a continuous function [MPNTW04, PCH95, PCH97]. Estimation of continuity between two system variables involves realizing the mathematical definition of continuity in an algorithm. To this end, one can use the algorithm developed in [MPNTW04]. This algorithm first focuses on a local region defined by a single fiducial point on one attractor (designated the source) and including its near neighbor points, and the corresponding neighborhood on the other attractor (designated as the target) defined by points with the same time indices as those on the source. The continuity statistic reflects the degree to which the points in the region on the source attractor map to a local region on the target attractor (indicating

176

L.J. Moniz et al.

greater likelihood of continuity) or are instead widely scattered across the target attractor (indicating smaller likelihood of continuity). Computations are based on a number of fiducial points across the source attractor, and inference is based on average values of these continuity statistics. Continuity statistics are not expected to be the same in both directions (i.e., when the roles of target and source attractors are reversed), reflecting any asymmetries in connectivity between the two system components. Mutual prediction [SSCBS96] is another approach to investigation of coupling that is based on reconstructed attractors from two time series. The approach assesses the degree to which dynamics of one attractor can be used to predict the dynamics of another. If the two attractors indeed belong to the same system, then their dynamics should follow similar dynamical paths. A fiducial point is randomly selected from one attractor, and the neighborhood local to this point is selected from the other attractor. This neighborhood is based strictly on spatial proximity, and there is no necessary relationship between the time indices of the fiducial point and the points in the corresponding neighborhood on the other attractor. The trajectories of these neighborhood points are then used to forecast the dynamics on the original attractor, and the difference between predicted and actual dynamics provides a metric reflecting predictive ability. As with continuity, predictive ability is assessed for a large number of points across the attractor and an average value computed. Mutual prediction can in fact be used as a test for continuity [SSCBS96]. Both continuity and mutual prediction between attractors can be asymmetric, reflecting differences in information flow between system components. This discussion has focused thus far on two state variables suspected to be components of the same system or network. We note here that it is also possible to build a multivariate attractor using information from multiple system state variables. It is then possible to use either continuity or mutual prediction to assess coupling of a single state variable and the multivariate reconstruction based on a number of other system components [NMNPC05]. These methods and others to be described in this section have seen little use in ecological settings; so we will illustrate some of them using time-series data generated from a two-species, spatially distributed, ecological model. The predator–prey model was introduced by Pascual [Pas93] and further explored by Little et al. [LEPNKSCS96] and Nichols et al. [NMNPC05]. The model describes system dynamics via dimensionless variables for predator .h/ and prey density .p/ along a one-dimensional spatial gradient, with location designated as x. System dynamics are specified as follows: @p ap @2 p D rx p.1 p/ hCd 2; @t 1 C bp @x @h ap @2 D h mh C d 2 h; @t 1 C bp @x rx D e f x:

(8.1)


177

Fig. 8.1 Mutual prediction computed for predator–prey dynamics at different spatial cell locations of a spatially extended system (8.1), with prey resource gradient extending from low (cell x D 1) to high (x D 0) resources with resource slope parameter f D 1:4. Predator–prey data from one model cell location are used to predict predator–prey dynamics at another location (predicted cell location). Scaled predictive ability is indicated by color and ranges from high (0, blue) to low (1, red)

Reflective boundary conditions are assumed at x D 0;1 with: @p @h D D 0: @x @x Parameters include predator–prey coupling a, prey-carrying capacity b, predator death rate m, diffusion coefficient d , and the intrinsic growth rate of the prey population rx , which is a function of space and reflects a linear gradient in prey resource abundance. Some parameter values were fixed as in [Pas93], m D 0:6, d D 104 , e D 5:0, and b D 2:0. This model is continuous with respect to time and system state (predator and prey abundances are real-valued rather than integers), and will thus provide reasonable approximations in some situations and not in others [DS04]. However, we note that the methods we describe should be applicable to discrete time and/or state models as well. The spatial network of predator and prey density is particularly suited to the following methods that describe connections within the network of observed variables. To illustrate continuity and mutual prediction, the above described model was integrated for nD1;000 time steps at spatial locations xi D 0:01; 0:02; : : : ; 0:99; 1:00 using resource gradient slope of f D 1:4. The dynamics for lattice site xi are given in state space by the vector .i / .i / .i / Xt D .Pt ; Ht /. We assess both the mutual prediction (Fig. 8.1) and continuity .i / .j / (Fig. 8.2) metrics between state vectors Xt ; Xt i; j D 0:01 : : : 1:00. Both approaches are clearly capable of identifying asymmetries in the coupling among the various lattice sites. In general, there is stronger evidence of continuity

178

L.J. Moniz et al.

Fig. 8.2 Continuity computed for predator–prey dynamics at different spatial cell locations of a spatially extended system (8.1), with prey resource gradient extending from low (cell x D 1) to high (x D 0) resources with resource slope parameter f D 1:4. The continuity statistic reflects the function relating predator–prey dynamics at one location (source) to dynamics at another location (target). Scaled continuity is indicated by color and ranges from high (0, blue) to low (1C, red)

mapping dynamics at the low end of the resource gradient (x D 1:00) to those at the high resource end (x D 0:00) than vice versa. Similarly, the dynamics at the low resource end do a better job (on average) of predicting dynamics at the high resource end. By highlighting asymmetries in spatial coupling, these results suggest a possible monitoring strategy (see later discussion and Nichols et al. [Njm05, NMNPC05], Moniz et al. [MNN07]). The results in Figs. 8.1 and 8.2 are based on attractors reconstructed using the time series of both predator and prey state variables. However, if only data from a single-state variable are available, results of Takens theorem can be used to reconstruct the attractor using delay coordinates to obtain similar results [NMNPC05]. This possibility leads to interesting questions about optimal allocation of resources to monitoring programs (e.g., one or more species at one or more locations).

Information-Theoretic Approaches Although attractor-based approaches are potentially useful in exploring coupling in dynamical systems, assessment of the direction of information flow is arguably ad hoc. Alternative approaches based on information theory have been recommended for the investigation of coupling, in part because such approaches involve formal characterization of the direction of information flow. Because these approaches have seen little use in ecology and because of our belief that they hold promise, we will describe two such information theory-based approaches in somewhat more detail than methods presented above (we discuss the joint utility of attractor-based and


179

information-theory approaches later). Let Y and Z represent two state variables reflecting two dynamical processes, and assume that we are investigating the possibility that they are coupled and actually components of the same system. We obtain simultaneous measurements of each system resulting in two time series, yt and zt , where t D 1 : : : M is a discrete time index. Each measurement is viewed as a discrete random variable with underlying probability distribution function p.yt / and p.zt /, respectively, and joint probability distribution p.yt ; zt /. The amount of information (in bits) about one state variable that is gained by knowing the value of the other variable is given by mutual information [VSw88, Wil97]: I.Y; Z/ D

X y;z

p.yt ; zt / log2

p.yt ; zt / : p.yt /p.zt /

(8.2)

Mutual information (8.2) is a Kullback entropy [Kul59] in that it reflects the excess information needed to encode p.yt ; zt / when erroneously assuming that the two variables are statistically independent [Sch00]. In order to compute mutual information, the probability distribution functions may be estimated using “bin counting” or other kernel density estimation approaches (e.g., see [Njm05, VSw88, Wil97]). The directionality of information flow or transport may be addressed by modifying the above expression for mutual information to include a time delay in one of the variables [Njm05, VSw88]: I.Y; ZT / D

X y;z

p.yt ; zt CT / log2

p.yt ; zt CT / ; p.yt /p.zt CT /

(8.3)

where T indicates the delay. This quantity is referred to as time delayed mutual information. The investigator searches for the delay, Tmax , for which I.Y; ZT / is a maximum. Tmax > 0 suggests information transport from Y to Z, whereas Tmax < 0 suggests information transport from Z to Y . When the system of interest is characterized by a spatial component, and when Yt and Zt CT can be measured at different points in space, then the functional relationship between Tmax and distance separating the two locations can even be used to draw inferences about the nature or form of the coupling function [VSw88, Njm05]. A more formal approach to inference about information flow was recently suggested by Schreiber [Sch00], who recommended a focus on state transition probabilities rather than on static probabilities. This approach, transfer entropy, considers systems that can be characterized as stationary Markov processes of specified order, k, where transition .k/ probabilities are denoted as p.yt C1 j yt ; yt 1 : : : yt kC1 / D p.yt C1 j yt /. The approach then considers another possible system state variable, Z, and asks whether knowledge of this variable provides additional information about the dynamics of Y . In the absence of information flow between Z and Y , the following equality should hold: .k/ .k/ .l/ p yt C1 j yt D p yt C1 j yt ; zt : (8.4)

180

L.J. Moniz et al.

Transfer entropy, TZ!Y , is a Kullback entropy focused on the deviation of the system from the generalized Markov property (8.4), i.e., on the extent to which extra information about the dynamics of Y is provided by Z: TZ!Y

.k/ .l/ p yt C1 j yt ; zt X .k/ .l/ : D p yt C1 ; yt ; zt log2 .k/ p yt C1 j yt yz

(8.5)

Transfer entropy is not symmetric, as information flow can be much stronger in one direction than another. As with attractor-based approaches, the above descriptions of time-delayed mutual prediction and transfer entropy considered two variables, but it is possible to develop multivariate analogs. For example, mutual information can be used to consider the additional information provided by one variable about the multivariate distribution of a number of other system variables [PT95]. Similarly, transfer entropy can be used to assess the additional information provided by one variable about the transition probabilities of a number of component state variables. Perhaps the biggest drawback to information-theoretic approaches is that their estimation can be difficult. Estimation of probability densities from time-series data is generally challenging, as the results will often depend considerably on how the data are “binned.” Kernel density estimates tend to provide good results and are used in most work on information-theoretic approaches (in this work as well). A good discussion of the estimation of both mutual information and transfer entropy can be found in [KS02]. In the cited work, the convergence of kernel-based estimators of both quantities is discussed. Of the two quantities, transfer entropy is the more difficult to estimate and no guarantees can be made regarding convergence. An alternative estimator of transfer entropy was proposed in [MK02] which purportedely works well for limited data. Reliable estimation will likely remain the core issue regarding implementation of information-theoretics in ecological applications for some time. As an example of the potential utility of IT approaches, consider the predator– prey model of (8.1) for resource gradient slope of f D 1:4. Using the time-delayed mutual information, we might explore how information (e.g., about predator/prey abundance) moves or flows from one spatial location to another. Let P xD0:96 denote the prey time series recorded at spatial location x D 0:96. Following the work of Vastano and Swinney [VSw88], we can examine I.P xD0:94 I PTxD0:96 / to examine how information moves from one lattice site to the other. Figure 8.3 shows the time-delayed mutual information computed between time series from several lattice sites at various levels of spatial separation. The “target” lattice site was fixed as x D 0:96 for each case. As shown in the plot, the “source” lattice site varied between x D 0:50 and x D 0:94. Several observations can be made from Fig. 8.3. First, the dominant peak of the mutual information occurs for positive lags indicating that information is moving from the high resource end to the low resource end [FS86].


181

Fig. 8.3 Time delayed mutual information, I.P x I PTxD0:96 /, for prey populations recorded at a target location (x D 0:96) and various other locations (x D 0:70; 0:75; 0:80; 0:85; 0:90; 0:94). Results based on the spatially extended predator–prey model of (8.1) with resource gradient characterized by slope f D 1:4

Fig. 8.4 Average transfer entropy from each site on the resource gradient to all other sites for three resource gradients, f D 0:9 (blue), f D 1:4 (red), f D 1:9 (magenta), where zero indicates high resources and one indicates low resources [model of (8.1)]. Solid lines indicate average Tprey!predator , whereas dashed lines indicate average Tpredator!prey

In addition, the peak occurs at larger and larger lags as the degree of spatial separation increases (information takes longer to travel larger distances), providing inference about the speed of information transport, and thus the rate of prey dispersal [Njm05]. Similarly, we can use the transfer entropy to assess directionality of information transport. Figure 8.4 shows plots of average transfer entropy from each

182

L.J. Moniz et al.

site to all other sites for three values of slope of the resource gradient, f D 0:9, f D 1:4 , f D 1:9 [see (8.1)]. The two plots for each slope show: (1) The average TEH x !P x0 ; reflecting the extra information about prey transitions (averaged over all locations x 0 ¤ x) provided by knowing the predator abundance at location x (solid plots). (2) The average TEP x !H x0 ; reflecting the extra information about predator transitions (averaged over all locations x 0 ! x) provided by knowing the prey abundance at location x (dashed plots). Two general comments emerge about information flow in this spatially extended predator–prey system. First, on average, more information about system dynamics is provided by knowledge of predator or prey abundance toward the low-resource end of the resource gradient. This result is consistent with the observations made previously using the continuity, mutual prediction, and mutual information metrics. Second, at locations near the high-resource end of the gradient, prey provide more information about predator dynamics than vice versa. Near the low-resource end of the plots, predators provide more information about prey, especially for the two steepest resource gradients, f D1:4, f D1:9. Results of this sort have clear implications for monitoring programs and concepts such as that of indicator species. Combining Attractor-Based and Information-Theoretic Approaches As noted above, inference about directionality of information flow based on attractor-based approaches is indirect, whereas the information-theoretic transfer entropy metric was developed specifically to assess directionality of information flow. However, transfer entropy does not always yield unambiguous inference either. For example, values of transfer entropy that approach zero [i.e., when .k/

.l/

p.yt C1 jyt ;zT / .k/ p.yt C1 jyt

! 1] can arise from two distinct situations [MNN07]. One possibility

is that the two systems are unrelated, such that zt is completely independent of yt . The other possibility is that the systems governing zt and yt are identical, such that knowledge of yt is equivalent to knowledge of zt . In the latter situation, information about zt may be useful in a monitoring context, whereas in the former case, information about zt should not be useful. Moniz et al. [MNN07] have proposed the use of continuity statistics to resolve this ambiguity.

3.2 Comparative Dynamics Methods for comparative dynamics focus on the detection of differences in dynamics of two systems based on two or more time series. Frequently, the question of interest will involve a single system that is observed for some initial period of time. Then, the system is perturbed, and, following a period of possible transient dynamics, a second time series is obtained. The question is whether the perturbation


183

has resulted in a change in system dynamics. The perturbation may be imposed experimentally or may instead involve natural phenomena. In some cases, the investigator may not identify a specific perturbation, but may instead be interested in possible damage or other changes that might occur as a system ages and is exposed to natural environmental variation. A number of questions about ecosystem change and damage are of interest to ecologists and natural resource managers. Both attractor-based and information-theoretic methods can be used to detect changes in system dynamics.

Attractor-Based Approaches Attractor-based approaches such as mutual prediction and continuity statistics can be used with two time series of the same state variable, representing periods before and after some perturbation of interest. If long-term system dynamics remain unchanged following any transients produced by the perturbation, then the mutual prediction algorithm and continuity statistics should reflect similar attractor geometries, whereas changes in dynamics should lead to dissimilar attractor geometries. “Recurrence plots” [EKR97] were developed for the purpose of detecting and exploring nonstationarities in time series. They represent a graphical technique designed to highlight structure by focusing on the number of times that a system returns to a dynamical state. Assume a dynamical system comprised of r state variables, xt , that is directly measured at M points in time, tD1 : : : M or obtained via attractor reconstruction. The recurrence matrix, Rts (where s and t denote different points in time), is simply a matrix of 0s and 1s with entries defined as Rts D1 when xt and xs are close together (where “close” is defined as k xt xs k< , where is a threshold parameter and k : k takes the Euclidean norm of the rdimensional distance vector), and Rts D 0 for jjxt xs jj > . It was demonstrated in [RO2008] that the recurrence matrix can be related to a thresholded version of the local covariance matrix. The selection of the threshold will depend on the application. Too small a threshold and no recurrences will be visible (the plot will be all white except for the diagonal) while too large a threshold includes all points in the signal thus obscuring any structure present in the local covariance. A reasonable rule of thumb that has worked well in several applications is to take D 0:1, where is the standard deviation of the time series. Unlike continuity and mutual prediction, recurrence plots do not require preservation of the true underlying dynamics in the reconstructed attractor. Rather, recurrence plots simply allow the practitioner to visualize local covariances in the data as a function of time (for stationary, ergodic processes, summing Rts over s and dividing by the number of data points, M; gives a probability density estimation of the system local to point “t”). Because recurrence plots are simply probing local density structure we do not have to faithfully reconstruct an “attractor” or even assume one exists. Recurrence plots are always populated on the main diagonal and symmetric about it for constant . Analysis of recurrence plots is based on diagonal and vertical line

184

L.J. Moniz et al.

structures, where a line is defined as l > 1 adjacent points with no intervening white (nonpopulated) spaces [GC00, IB98, MWSK02, TGZW96]. Vertical lines reflect traditional ideas of autocorrelation, whereas diagonal structures reflect deterministic dynamics. Some kinds of purely stochastic dynamics (e.g., Gausian white noise) are characterized by the absence of structure, as nearby points at one time, t, have little chance of being neighbors again the next time, t C 1. Recurrence plots can be modified to deal specifically with comparative dynamics of two processes, x.n/ and y.n/, by forming a cross-recurrence matrix [NTS06] based on the distance between points in the state space. With this approach, CRts D 1 for k xt ys k< , and CRts D 0 for k xt ys k> . In the case of cross-recurrence plots, line structures relate to the probability that the two systems obey the same dynamics. In the case of ecosystem change, the two processes could represent the same state variable(s) before and after a perturbation. Recurrence quantification analysis [WZ94] refers to the computation of various summary statistics from recurrence and cross-recurrence plots or matrices. For example, percent recurrence is the percentage of darkened points in the plot, whereas percent determinism is the percentage of darkened points occupying significant line structures, where significant is often just taken to mean two or more adjacent points. These statistics can then be compared for recurrence plots made from time series before and after a perturbation, or statistics can be computed directly for the cross-recurrence plots based on the two time series. As an example, again consider the predator–prey model described in (8.1). Figure 8.5 shows recurrence plots generated from the prey dynamics at spatial locations 0.13 (upper plot) and 0.96 (lower plot). Both plots were generated using a recurrence length scale of D 0:1 where is the standard deviation of the time series.The dynamics at location 0.13 are largely periodic, hence the banded structure of the recurrence plot. The period of the oscillation corresponds directly to the spacing between the bands. Prey dynamics at location 0.96 are chaotic [Pas93] and exhibit a more complicated recurrence (probability) structure. If the diffusion constant d is modified to include temporal fluctuations, then a very different recurrence structure emerges. Letting d.t/ D d C 5.105 sin.2gt / /, where g D 0:001 samples/unit time (slow periodic modulation of diffusion coefficient), one sees different recurrence patterns (Fig. 8.6). The continuous, periodic bands at lattice site 0.13 are replaced by wavy patterns that are clearly being modulated by the time dependency in the diffusion term. The local probability density structure at lattice site 0.96 is also altered by the influence of d . However, in this case the complexity of the original recurrence plot makes it difficult to detect the influence of the nonstationarity in the diffusion term. This simple example illustrates the types of changes that can be observed when examining recurrence structure in time series data.

Information-Theoretic Approaches Time-delayed mutual information and transfer entropy were described above as methodological approaches for the detection of coupling and information flow


185

Fig. 8.5 Recurrence plots (both axes reflect time) of prey dynamics at spatial location 0.13 (upper) and location 0.96 (lower) along a spatial resource gradient from high (0) to low (1) prey resources. Recurrence length scale is D 0:1 , where is the standard deviation of the time series. Dynamics are based on the predator–prey model of (8.1), with constant diffusion coefficient, d

between system components. Non-transient changes in system dynamics following a perturbation to the system should be reflected in changes in coupling and information flow among system components. In the case of ecological monitoring, the two state variables or components might be two different species at the same location(s) or the same species at two different locations that are sufficiently close to be dynamically interdependent (see [Njm05, NMNPC05]). The approach to change detection would involve computation of time-delayed mutual information or transfer entropy for two system components during the period before, and then following,

186

L.J. Moniz et al.

Fig. 8.6 Recurrence plots (both axes reflect time) of prey dynamics at spatial location 0.13 (upper) and location 0.96 (lower) along a spatial resource gradient from high (0.00) to low (1.00) prey resources. Recurrence length scale is D 0:1 , where is the standard deviation of the time series. Dynamics are based on the predator–prey model of (8.1), with diffusion coefficient varying temporally according to d.t / D d C 5.105 sin.2gt /, where g D 0:001 samples/unit time (slow periodic modulation of diffusion coefficient)

a perturbation, where the post-perturbation data come after a delay to allow for transient dynamics. Differences in these metrics would reflect changes in system dynamics, whereas similarities would indicate an absence of change, at least with respect to the selected system components. This section on comparative dynamics has focused on the comparison of one or more time series at two different times (e.g., before and after a perturbation).


187

We view this approach as potentially the most useful in assessment of ecosystem change. However, we also note that comparative dynamics may sometimes involve comparison of observed dynamics with that expected under a null hypothesis of interest. For example, Nichols et al. [NSTSP06] have proposed the use of the described information-theoretic approaches for damage detection in the field of structural health monitoring. One approach to this problem would be to compare a structure before and after the occurrence of suspected damage. However, [NSTSP06] note the advantages of being able to assess damage in the absence of baseline or pre-damage data. They equate nonlinearity with damage and then test for nonlinearity as a means of testing for damage. Specifically, they take time series of two system components from a structure that may be damaged (in their case, sensors placed at two different locations on the structure). They then create surrogate data sets from these original data sets that retain the exact linear cross correlation between the two measured variables. However, randomization is used to destroy higher-order correlations that may exist, thus producing surrogates that represent the null hypothesis of linearity. Comparison of this null hypothesis with the alternative of nonlinear dynamics is accomplished using time-delayed mutual information and transfer entropy, and damage (nonlinear coupling) is readily detected using this approach [NSTSP06]. The relevance of this approach depends on the reasonableness of equating nonlinearity and damage. This view of damage as nonlinearity is very common in engineering applications, but is also held by some in ecosystem thinking [Patt75].

4 Surveillance Monitoring and Information Extraction In the following discussion, we return to the theme that the described methods of extracting information from time-series data form the basis of a conceptual framework for surveillance monitoring. We provide a linkage between the described methods and ecological inquiry, considering topics and questions of interest to ecologists, and suggesting specific methods that may be useful in addressing them.

4.1 System Identification Ecological systems represent “enormously complex associations of species which interact in diverse ways. As a matter of practical necessity, field ecologists can rarely specify, much less quantify, all of the interactions” ([Sch81], p. 383). In the face of such unknowable complexity, ecologists have chosen to monitor the dynamics of one or a few species in hopes of learning something about the entire system. As noted above, Takens [Tak81] embedding theorem (also see [Yul27, Whi36, PCFS87, SY91, OY03, PMNC07]) provides a theoretical basis for

188

L.J. Moniz et al.

such hopes by demonstrating the possibility of learning about the dynamics of an entire system using a time series of data from a single (or small number of) system state variable(s). We noted that attractor reconstruction using delay coordinates provides a geometric representation of system dynamics that provides information about the portions of state space in which we expect to find the system most of time. Such reconstructions also provide a basis for computing metrics (e.g., mutual prediction, continuity) for inference about coupling of system components.

4.2 Indicator Species Ecologists have long recognized that presence or dynamics of a single species or group of species can be used as indicators of both physical and biological characteristics of systems in which they are found ([Odu71] and references therein). This recognition has evolved to the point where “indicator species” have become a central concept in ecological monitoring. The basic premise is that monitoring all system components is impossible for all but the simplest systems; so some selection of components to monitor is required [MZSM04, Sim98]. The literature of ecology and conservation biology contains many discussions of indicator species including such topics as a rationale for their use, methods for their selection, and criticisms of the concept [LVT88, MZSM04, Nos90, Noo03]. Noon ([Noo03], p. 43) specifies that indicator species should “provide information on the state of the unmeasured resources and processes of the focal ecological system”, and emphasizes that “the ultimate success or failure of the [monitoring] program may be determined by this one step.” We believe that Takens [Tak81] theorem provides a theoretical justification for the concept of indicator species, insofar as it demonstrates the potential for time series of a single species to be used to reconstruct (“indicate”) the dynamics of the system over the entire state space. Furthermore, the described methods for assessing coupling and dynamical interdependence provide a natural framework for considering indicator species as well as objective approaches for their selection. Both attractor-based methods (continuity and mutual prediction) and informationtheoretic methods (time-delayed mutual information and transfer entropy) can provide asymmetric estimates of coupling, reflecting differences in information flow between either two measured system components or between one system component and a multivariate set of other components. The general work of Pecora et al. [PMNC07] on selection of time-delayed versions of system state variables for attractor reconstruction is very relevant to the selection of a set of indicator species that provides the most information about the dynamics of the entire system. For model systems designed to mimic real-world ecosystems, single state variables could be investigated one at a time for their information content with respect to dynamics of the entire system. Such an exercise could, for example, provide


189

general inferences about trophic level (our example use of transfer entropy for a predator–prey system) or other characteristics (e.g., generalists or specialists) of species having relatively high and low information content. The methods can also be used, or adapted for use, with actual monitoring data for example systems to provide inferences about which monitored species is providing the most information about the studied system. It would also be possible to compute the loss in information incurred by eliminating species from a list of those monitored. We believe that transfer entropy may hold the greatest promise for the investigation of indicator species, although we certainly do not rule out any of the described approaches that focus on coupling.

4.3 Species Interactions The discipline of community ecology focuses on interactions between pairs of species. Trophic interactions, competitive interactions, and mutualistic interactions are all of interest to ecologists, yet they are not always readily observable. In some situations, it may be useful to pose questions about dynamically important interspecific interactions using time-series data from monitoring programs [IDCC03]. The described methods for investigation of coupling may be useful in such situations. Time series of two species suspected to interact can be investigated using attractorbased (continuity statistics, mutual prediction) or information-theoretic (mutual information, transfer entropy) approaches. Resulting inferences about coupling will be indicative of either dynamic interactions or dynamic responses to the same driving variable, a dichotomy that can be resolved in some cases using approaches such as transfer entropy and time-delayed mutual information [Njm05]. Asymmetric interactions between species are especially relevant to some ecological hypotheses [VSi02], and the ability of the described methods to detect and quantify asymmetric coupling will thus be useful. Initial efforts to draw inferences about simple food web structure based solely on time-series data from web components have been encouraging [MCENN07].

4.4 Spatial Coupling, Population Synchrony, and Spatial Sampling In addition to investigating possible interactions among different species at the same locations, ecologists are interested in possible coupling of populations of the same species at a network of different spatial locations. Time series of monitoring data from different locations are frequently used to draw inferences about population synchrony [BIL99, CS03, Caz04, Koe99]. When evidence of such synchrony is found, two general classes of underlying mechanisms are typically

190

L.J. Moniz et al.

considered [BIL99, Koe99, RKL98, Njm05]. One involves active dispersal, with animals moving between locations. The other possibility involves response to a common environmental driver [Mor53]. Both attractor-based and information-theoretic approaches have been shown to be useful in assessing spatial coupling using time series generated from a spatially extended predator–prey model. Nichols et al. [NMNPC05] used both continuity and mutual prediction statistics to detect coupling and to draw inferences about its relative magnitude. Asymmetric spatial coupling was observed and was interpreted in terms of a prey resource gradient, and resultant spatial variation in dynamics. A comparison of these results with results based on linear cross-correlation led to the conclusion that the nonlinear approaches were superior for this model system. Nichols [Njm05] then used time-delayed mutual information with this same model system to draw more formal inferences about information flow between spatial locations within this system. He was further able to reject the hypothesis that a common environmental driver was responsible for the interdependent dynamics and even drew inferences about the nature of the dispersal functions responsible for the coupling [Njm05]. Mutual information has been used in conjunction with surrogate data sets to detect population synchrony in two-patch model systems and in actual data sets [CS03, Caz04]. In addition to investigating the factors responsible for dynamics of spatially extended ecological systems, these methods should be useful in providing insight into the relative value of different spatial sampling designs. Despite the abundance of work on static designs for spatial sampling [Tho02], there has been relatively little consideration of designs for sampling dynamical systems that exhibit variation over both time and space (see [MNN07, WR99] for exceptions). Stated differently, whereas ecologists have expended much effort discussing the concept of indicator species, there has been little attention devoted to consideration of possible indicator locations, locations that provide maximum information about dynamics of the system. As we noted, it is possible to use mutual information and transfer entropy to directly assess the information flow between multiple state variables (e.g., the abundances of a species at multiple locations within a system) and a single state variable (e.g., abundance of one species at one location) and vice versa [MNN07]. It is possible to use these methods on data from model systems with resource gradients or gradients in abundance, to try to gain insight into the characteristics of locations that provide most information about system dynamics. For example, the results presented above for the spatially extended predator–prey system based on the four different coupling metrics (continuity, mutual prediction, time-delayed mutual information, transfer entropy) indicated greater information flow from locations of high to low prey resource abundance than vice versa, a result consistent with the greater number of dispersers going from areas of high abundance and resources to areas of low abundance and resources [NMNPC05, Njm05]. This flow of information leads to an asymmetry in the information content of time series from different locations, such that low-resource locations provide more information about high-resource locations than vice versa (Figs. 8.1–8.4). This general inference based on multiple locations and different resource gradients is consistent with the recent


191

result of Jonzen et al. [JRP05] that in source-sink systems (sensu Pulliam [Pul88]), with dispersal from the source location to the sink but not vice versa, it is often most efficient to monitor the sink habitat. These results lead to the natural, yet henceforth unexploited, suggestion that asymmetric dispersal may be an important determinant of information content of monitoring data from various locations in ecological systems (also see [MNN07]). It may also be possible to directly determine the most information-rich locations using data from spatially extended monitoring programs. Of course, the sampling design problem can be treated more generally as one of finding the specieslocation combinations that provide the most information about system dynamics (see [NMNPC05]). This combination of ideas about indicator species and spatial sampling points to a potentially productive approach of treating surveillance monitoring as a joint optimization problem.

4.5 Environmental Drivers Our discussion thus far has focused on coupling between system components, whereas we are also frequently interested in environmental variables that may drive system dynamics. Pascual and Ellner [PE0] have developed approaches for identifying the driving variable from a set of candidates using time series of both potential drivers and system state variables. They focused on the period of the environmental driver and noted that more general approaches should be possible. All of the methods described above for assessment of coupling should be useful for this purpose. The information-theoretic approaches seem especially well suited for identification of environmental drivers, and indeed Nichols [Njm05] identified a periodic driver for a model predator–prey system using mutual information. Cazelles [Caz04] has used information-theoretic approaches in conjunction with symbolic dynamics and the generation of surrogate data to investigate coupling in ecological time series. He demonstrated superiority of this approach to linear cross-correlation in model systems. He also used the approach successfully with actual ecological time series and found evidence of an association between sheep abundance on the island of St. Kilda and the North Atlantic Oscillation index, a composite environmental variable [Caz04].

4.6 Assessment of Ecosystem Change and Damage Ecologists are frequently interested in comparing time series of state variables collected before and after some perturbation of interest (e.g., environmental change, management action) and even during two time periods not separated by some discrete event. The question is simply, have system dynamics been altered or do they

192

L.J. Moniz et al.

remain the same in the two periods? Mutual prediction provides one approach to investigating change, as prediction from one attractor to the other should be possible only if the system dynamics in the two time periods remain similar. Similarly, recurrence plots for the two time series should differ if dynamics have changed. Another approach to detection of change involves the assessment of coupling between two state variables before and after the perturbation. Changes in system dynamics are reflected in changes in coupling of system components. Indeed, engineers place multiple sensors on structures and use evidence of changes in coupling between locations to detect damage in structural health monitoring [NNTSTV04]. Thus, we can envision using either attractor-based or informationtheoretic approaches to assess coupling between two state variables before and following a perturbation. Finally, we noted above that sometimes the described methods can be used with a single time series, or with time series of two state variables from the same time period, to draw inferences that may be relevant to change or damage. Recurrence plots have been used to detect nonstationarity of time series, and this approach could be used with ecological data, as in the above predator–prey example. It was also noted above that structural damage is frequently equated with nonlinear responses to vibration. It is thus possible to generate surrogate data for two coupled time series under the null hypothesis of a linear relationship, and to test this against the alternative of nonlinearity [NTS06]. Some ecologists view nonlinearity as a signal of ecosystem damage [Patt75], although we doubt that this will generally be true. However, if the distinction between damaged and undamaged is better captured by some other contrast in dynamical pattern, it may be possible to design surrogates that reflect other null hypotheses for testing [SS2000].

5 Summary We began this review with three basic observations. First, ecological monitoring programs are of two basic types with respect to design and methods of analysis: hypothesis-driven programs developed for the conduct of science or management and surveillance programs developed as general sources of ecological information. Second, many ecological monitoring programs in place today are best characterized as surveillance monitoring. Third, virtually all of the methodological development associated with monitoring program design and data analysis has occurred for hypothesis-driven programs, with little quantitative attention having been devoted to surveillance monitoring. In this review, we have viewed the analysis of data from surveillance monitoring programs as an exercise in information extraction and as a specific example of the more general problem of determining interactions, information flow and synchronization from observations of an ecological network. We have reviewed methods developed primarily in other disciplines for analyzing time series and suggested that they provide a natural methodological framework for surveillance monitoring programs. In some cases, the methods provide an upper


193

limit to what can be learned from time-series data. In other cases, they can be used with ecological models to design surveillance monitoring programs. We believe that such investigations hold promise for possibly drawing general inferences about information content of monitoring data from different classes (e.g., trophic levels) of species and different locations characterized by different levels of immigration and emigration. We also believe that work with model systems can be used to explore the idea of surveillance monitoring design as a joint optimization problem involving both species and space. Finally, in still other cases the methods described in this review should be directly useful for analysis of certain kinds of ecological data. Some of the described methods can be used in the general process of system identification. Other methods are useful for identifying important environmental drivers and for assessing the strength of species interactions. If multiple locations are monitored, then spatial coupling of populations can be identified, and general inferences about dispersal are even possible. Some methods are well suited to investigating possible changes in long-term system dynamics occurring either generally over time or in association with an identified event. The attractor-based approaches discussed above were developed for long time series (many thousands of points) from deterministic systems, whereas the information-theoretic approaches appear to be more flexible and widely applicable. Three practical issues that arise when considering application of these methods to ecological time series are short series lengths, stochasticity, and stationarity. We believe that much can be learned from model systems for which none of these issues should be a problem. In addition to such exercises, time series can be generated from models, and the relevance of stochasticity and series length to inferences resulting from the described methods can be directly assessed. Recent efforts of this type have been encouraging, with information-theoretic approaches being useful with short time series and in the presence of stochasticity [MCENN07]. Several approaches are available for assessing system stationarity and have been used for identifying ecological time series appropriate for the analyses described herein [MNN07]. Nevertheless, for actual ecological monitoring data, much work remains. With respect to series length, some ecological time series (e.g., meteorological variables such as air temperature and wind speed, water levels in certain systems) are sufficiently long for attractor-based approaches [KSE04, MNN07]. Time series of animal abundances do not tend to be nearly long enough for some of the described approaches, and the likely solution involves more specific parametric modeling [ET95, TE00] than is necessary for long series. The use of null hypothesis surrogate data sets has also proven useful with short data sets arising in ecology and other disciplines [CS03, Caz04, MK02]. With respect to the issue of determinism and stochasticity, the recent assessment of Schreiber ([Sch00], p. 3) seems relevant. “Neither naive enthusiasm to explain all kinds of unsolved time-series problems by nonlinear determinism is justified, nor is the pessimistic view that no real system is ever sufficiently deterministic and thus out of the reach for analysis. At least, chaos theory has inspired a new set of useful time-series tools and provides a new language to formulate time-series problems – and to find their solutions.”

194

L.J. Moniz et al.

Acknowledgments We acknowledge the support of Paul Dresler and US Geological Survey inventory and monitoring program for research on these topics.

References [ABB06] [BJP09] [BIL99]

[Bou01] [Bou96] [CGC06]

[Cas92]

[Caz04] [CS03] [CT92] [CCDD95]

[CDCD97] [CCDDH03]

[DDCC95]

[DDCC97]

[DS04] [EKR97] [ER95] [ET95]

Allesina, S., Bodini, A., Bondavalli, C.: Ecological subsystems via graph theory: the role of strongly connected components. Oikos 110, 164–176 (2005) Bauer B., Jordan, F., Podani, J.: Node centrality indices in food webs: rank orders versus distributions. Ecol. Complexity (in press) Bjornstad, O.N., Ims, A.R., Lambin, X.: Spatial population dynamics: analyzing patterns and processes of population synchrony. Trends Ecol. Evol. 14, 427–432 (1999) Boudjema, G., Cazelles, B.: Extraction of nonlinear dynamics for short and noisy time series. Chaos Solutions Fractals 12, 2051–2069 (2001) Boudjema, G., Chau, N.P.: Revealing dynamics of ecological systems from natural recordings. Ecol. Model. 91, 15–23 (1996) Cartozo, C.C., Garlaschelli, D., Caldarelli, G.: Graph theory and food webs, In: Pascual, M, Dunne, J.A. (eds.) Ecological Networks, SFI Studies in the Sciences of Complexity. Oxford University Press, Oxford, UK (2006) Casdagli, M.: A dynamical systems approach to modeling input-output systems, In: Casdagli, M., Eubank. S, (eds.) Nonlinear Modeling and Forecasting, SFI Studies in the Sciences of Complexity, vol. 12, pp. 265–281. AddisonWesley, Reading, Massachusetts (1992) Cazelles, B.: Symbolic dynamics for identifying similarity between rhythms of ecological time series. Ecol. Lett. 7, 755–763 (2004) Cazelles, B, Stone, L.: Detection of imperfect population synchrony in an uncertain world. J. Anim. Ecol. 72, 953–968 (2003) Cheng, B., Tong, H.: On consistent nonparametric order determination and chaos. J. Roy. Stat. Soc. B 54, 427–449 (1992) Constantino, R.F., Cushing, J.M., Dennis, B., Desharnais, R.A.: Experimentally induced transitions in the dynamic behaviour of insect populations. Nature 375, 227–230 (1995) Constantino, R.F., Desharnais, R.A., Cushing, J.M., Dennis, B.: Chaotic dynamics in an insect population. Science 275, 389–391 (1997) Cushing, J.M., Constantino, R.F., Dennis, B., Desharnais, R.A., Henson, S.M.: Chaos in ecology: experimental nonlinear dynamics. Academic Press, San Diego (2003) Dennis, B., Desharnais, R.A., Cushing, J.M., Constantino, R.F.: Nonlinear demographic dynamics: mathematical models, statistical methods, and biological experiments. Ecol. Monogr. 65, 261–281 (1995) Dennis, B., Desharnais, R.A., Cushing, J.M., Constantino, R.F.: Transitions in population dynamics: equilibria to periodic cycles to aperiodic cycles. J. Anim. Ecol. 66, 704–729 (1997) Domokos, G., Scheuring, I.: Discrete and continuous state population models in a noisy world. J. Theor. Biol. 227, 535–545 (2004) Eckmann, J.-P., Kamphorst, S.O., Ruelle, D.: Recurrence plots of dynamic systems. Europhys. Lett. 4, 973–977 (1987) Eckmann, J.-P., Ruelle, D.: Ergodic theory of chaos and strange attractors. Rev. Mod. Phys. 57, 617–656 (1985) Ellner, S, Turchin, P.: Chaos in a noisy world: new methods and evidence from time-series analysis. Am. Natur. 145, 343–375 (1995)

8 Inferences About Coupling from Ecological Surveillance Monitoring: Approaches: : : [Fis47] [Fis58] [FS86] [GC00] [HHETG93]

[HM97] [Hur84] [IDCC03]

[IB98] [JRP05]

[Jor09] [JOBL08]

[KS02] [KR02] [KSE04]

[Koe99] [Kul59] [LVT88] [LEPNKSCS96]

[MZSM04]

[MK02]

[MWSK02]

[May73] [Mil85]

195

Fisher, R.A., The Design of Experiments, 4th edn. Hafner, New York (1947) Fisher, R.A.. Statistical Methods for Research Workers, 13th edn. Oliver and Boyd, London (1958) Fraser, A.M., Swinney, H.L.: Independent coordinates for strange attractors from mutual information. Phys. Rev. A 33, 1134–1140 (1986) Gao, J., Cai, H.: On the structures and quantification of recurrence plots. Phys. Lett. A 270, 75–87 (2000) Hastings, A., Hom, C.L., Ellner, S., Turchin, P., Godfray, H.C.J.: Chaos in ecology: is mother nature a strange attractor? Ann. Rev. Ecol. Syst. 24, 1–33 (1993) Hilborn, R., Mangel, M.: The Ecological Detective. Confronting Models with Data. Princeton University Press, Princeton (1997) Hurlburt, S.H.: Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54, 187–211 (1984) Ives, A.R., Dennis, B., Cottingham, K., Carpenter, S.R.: Estimating community stability and ecological interactions from time-series data. Ecol. Monogr. 73, 301–330 (2003) Iwanski, J.S., Bradley, E.: Recurrence plots of experimental data: to embed or not to embed. Chaos 8, 861–871 (1998) Jonzen, N., Rhodes, J.R., Possingham, H.P.: Trend detection in source-sink systems: when should sink habitats be monitored? Ecol. Appl. 15, 326–334 (2005) Jordan, F.: Keystone species and food webs. Phil. Trans. Soc. B 364, 1733–1741(2009) Jordan, F., Okey, T.A., Bauer, B., Libralato, S.: Identifying important species: lining structure and function in ecological networks. Ecol Model. 216, 75–80 (2008) Kaiser, A., Schreiber, T.: Information transfer in continuous processes. Phys. D 166, 43–52 (2002) Keeling, M.J., Rohani, P.: Estimating spatial coupling in epidemiological systems: a mechanistic approach. Ecol. Lett. 5, 20–29 (2002) Kocak, K., Saylan, L., Eitzinger, J.: Nonlinear prediction of near-surface temperature via univariate and multivariate time series embedding. Ecol. Model. 173, 1–7 (2004) Koenig, W.: Spatial autocorrelation of ecological phenomena. Trends Ecol. Evol. 14, 22–26 (1999) Kullback, S.: Information Theory and Statistics. Wiley, New York (1959) Landres, P.B., Verner, J., Thomas, J.W.: Ecological uses of vertebrate indicator species: a critique. Con. Biol. 2, 316–328 (1988) Little, S., Ellner, S., Pascual, M., Neubert, M., Kaplan, D., Sauer, T., Caswell, H., Solow, A.: Detecting nonlinear dynamics in spatio-temporal systems, example from ecological models. Phys. D 96, 321–333 (1996) Manley, P.N., Zielinski, W.J., Schlesinger, M.D., Mori, S.R.: Evaluation of a multiple-species approach to monitoring species at the ecoregional scale. Ecol. Appl. 14, 296–310 (2004) Marschinski, R., Kantz, H.: Analysing the information flow between financial time series: An improved estimator for transfer entropy. Eur. Phys. J. 30, 275–281 (2002) Marwan, N., Wessel, N., Schirdewan, A., Kurths, J.: Recurrence-plot-based measures of complexity and their application to heart-rate-variability data. Phys. Rev. E 66, Article Number 026702 (2002) May, R.M.: Stability and Complexity in Model Ecosystems. Princeton University press, Princeton, NJ (1973) Milnor, J.: On the concept of attractor. Comm. Math. Phys. 99, 177–195 (1985)

196 [MCENN07]

[MNN07]

[MPNTW04]

[MP2008]

[Mor53] [Njd00]

[NW06] [Njm05]

[NMNPC05]

[NNTSTV04]

[NSTSP06]

[NTS06]

[Noo03]

[Nos90] [Odu71] [OY03] [PCFS87] [Pas93] [PE0] [Patt75]

L.J. Moniz et al. Moniz, L.J., Cooch, E.G., Ellner, S.P., Nichols, J.D., Nichols, J.M.: Application of information theory methods to food web reconstruction. Ecol. Model. 208, 145–158 (2007) Moniz, L.J., Nichols, J.D., Nichols, J.M.: Mapping the information landscape: discerning peaks and valleys for ecological monitoring. J. Biol. Phys. 33, 171–181 (2007) Moniz, L., Pecora, L., Nichols, J., Todd, M., Wait, J.R.: Dynamical assessment of structural damage using the continuity statistic. Int. J. Struct. Health Monit. 3, 199–212 (2004) Moniz, L., Peter, W.: Application of Nonlinear Data Analysis to Locating Disease Clusters. Proceedings of the 10th Experimental Chaos Conference Catania, Italy (2008) (to appear) Moran, P.A.P.: The statistical analysis of the Canadian lynx cycle. II. Synchronization and meteorology. Aust. J. Zool. 1, 291–298 (1953) Nichols, J.D.: Monitoring is not enough: on the need for a model-based approach to migratory bird management. In: Bonney, R., Pashley, D.N., Cooper, R., Niles, L. (eds.) Strategies for Bird Conservation: The Partners in Flight Planning Process, pp. 121–123. Proceedings RMRS-P-16. U.S.D.A., Forest Service, Rocky Mountain Research Station, Ogden, Utah (2000) Nichols, J.D., Williams, B.K.: Monitoring for conservation. Trends Ecol. Evol. 21, 668–673 (2006) Nichols, J.M.: Inferences about information flow and dispersal for spatially extended population systems using time-series data. Proc. Roy. Soc. B 272, 871–876 (2005) Nichols, J.M., Moniz, L., Nichols, J.D., Pecora, L.M., Cooch, E.: Assessing spatial coupling in complex population dynamics using mutual prediction and continuity statistics. Theor. Pop. Biol. 67, 9–21 (2005) Nichols, J.M., Nichols, C.J., Todd, M.D., Seaver, M., Trickey, S.T., Virgin, L.N.: Use of data-driven phase space models in assessing the strength of a bolted connection in a composite beam. Smart Materials Structures 13, 241–250 (2004) Nichols, J.M., Seaver, M., Trickey, S.T., Salvino, L.W., Pecora, D.L.: Detecting impact damage in experimental composite structures: an information-theoretic approach. J. Smart Materials Structures 15, 424–434 (2006) Nichols, J.M., Trickey, S.T., Seaver, M.: Damage detection using multivariaterecurrence quantification analysis. Mech. Syst. Signal Process. 20, 421–437 (2006) Noon, B.R.: Conceptual issues in monitoring ecological resources. In: Busch, D.E., Trexler, J.C. (eds.) Monitoring Ecosystems, pp. 27–71. Island Press, Washington, DC (2003) Noss, R.F.: Indicators for monitoring biodiversity: a hierarchical approach. Con. Biol. 4, 355–364 (1990) Odum, E.P.: Fundamentals of Ecology, 3rd edn. Saunders, W.B., Philadelphia (1971) Ott, W., Yorke, J.A.: Learning about reality from observation. SIAM J. Appl. Dyn. Syst. 2, 297–322 (2003) Packard, N.H., Cruchfield, J.P., Farmer, J.D., Shaw, R.S.: Geometry from a time series. Phys. Rev. Lett. 45, 712–716 (1987) Pascual, M.: Diffusion-induced chaos in a spatial predator–prey system. Proc. Roy. Soc. London B 251, 1–7 (1993) Pascual, M., Ellner, S.P.: Linking ecological patterns to environmental forcing via nonlinear time series models. Ecology 81, 2767–2780 (2000) Patten, B.C.: Ecosystem linearization: an evolutionary design problem. Am. Natur. 109, 529–539 (1975)

8 Inferences About Coupling from Ecological Surveillance Monitoring: Approaches: : : [PCH95]

[PCH97]

[PMNC07] [PSWM00] [PF02] [PT95] [Pul88] [RKL98]

[RO2008]

[Sau04] [SY91] [Sch81] [SEK86] [SG93] [SSCBS96]

[Sch99] [Sch00] [SS2000] [Sim98] [SR92] [SBDH97] [Sta99] [SBDH03] [Str94]

197

Pecora, L.M., Carroll, T.L., Heagy, J.F.: Statistics for mathematical properties of maps between time series embeddings. Phys. Rev. E 52, 3420–3439 (1995) Pecora, L.M., Carroll, T.L., Heagy, J.F.: Statistics for continuity and differentiability: an application to attractor reconstruction from time series. Fields Inst. Comm. 11, 49–62 (1997) Pecora, L.M., Moniz, M., Nichols, J.M., Carroll, T.L.: A unified approach to attractor reconstruction. Chaos 17, 013110–013110–9 (2007) Perry, J.N., Smith, R.H., Woiwood, I.P., Morse, D.R. (eds.): Chaos in Real Data. Kluwer, Boston (2000) Post, E., Forchammer, M.C.: Synchronization of animal population dynamics by large-scale climate. Nature 420, 168–171 (2002) Prichard, D., Theiler, J.: Generalized redundancies for time series analysis. Phys. D 84, 476–493 (1995) Pulliam, H.R.: Sources, sinks and population regulation. Am. Natur. 132, 652–661 (1988) Ranta, E., Kaitala, V., Lundberg, P.: Population variability in space and time: the dynamics of synchronous population fluctuations. Oikos 83, 376–382 (1998) Rohde, G.K, Nichols, J.M., Dissinger, B.M., Bucholtz, F.: Stochastic analysis of recurrence plots with applications to the detection of deterministic signals. Phys. D 237, 619–629 (2008) Sauer, T.: Reconstruction of shared nonlinear dynamics in a network. Phys. Rev. Lett. 93, 198701 (2004) Sauer, T., Yorke, J.A., Casdagli, M.: Embedology. J. Stat. Phys. 65, 579–616 (1991) Schaffer, W.M.: Ecological abstraction: the consequences of reduced dimensionality in ecological models. Ecol. Monogr. 51, 383–401 (1981) Schaffer, W., Ellner, S., Kot, M.: Effects of noise on some dynamical models of ecology. J. Math. Biol. 24, 479–523 (1986) Scheiner, S.M., Gurevitz, J. (eds.): Design and Analysis of Ecological Experiments. Chapman and Hall, New York (1993) Schiff, S.J., So, P., Chang, T., Burke, R.E., Sauer, T.: Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble. Phys. Rev. E 54, 6708–6724 (1996) Schreiber, T.: Interdisciplinary application of nonlinear time series methods. Phys. Rep. 308, 1–64 (1999) Schreiber, T.: Measuring information transfer. Phys. Rev. Lett. 85, 461–464 (2000) Schreiber, T., Schmitz, A.: Surrogate time series. Phys. D 142, 346–382 (2000) Simberloff, D.: Flagships, umbrellas, and keystones: is single-species management pass in the landscape era? Biol. Cons. 83, 247–57 (1998) Skalski, J.R., Robson, D.S.: Techniques for Wildlife Investigations. Academic Press, San Diego (1992) Stark, J., Broomhead, D.S., Davies, M.E., Huke, J.: Takens embedding theorem for forced and stochastic systems. Nonlinear Anal. 30, 5303–5314 (1997) Stark, J.: Delay embeddings of forced systems I: Deterministic forcing. J. Nonlinear Sci. 9, 255–332 (1999) Stark, J., Broomhead, D.S., Davies, M.E., Huke, J.: Delay embeddings of forced systems II: Stochastic forcing. J. Nonlinear Sci. 13, 519–577 (2003) Strogatz, S.H.: Nonlinear Dynamics and Chaos, with Applications to Physics, Biology, Chemistry, and Engineering. Addison-Wesley, Reading Massachusetts (1994)

198 [Tak81]

[Tho02] [TGZW96]

[Tur03] [TE00]

[VSw88] [VSi02] [WZ94]

[Whi36] [WR99] [WR05] [Wil82] [Wil89] [Wil96] [WNC02] [Wil97] [YNB01] [Yul27]

L.J. Moniz et al. Takens, F.: Detecting strange attractors in turbulence, In: Rand, D.A., Young L.S. (eds.) Dynamical Systems and Turbulence, vol. 898, pp. 366–381. Lecture Notes in Mathematics Springer, Berlin (1981) Thompson, S.K.: Sampling. Wiley, New York (2002) Trulla, L.L., Giuliani, A., Zbilut, J.P., Webber, C.L., Jr.: Recurrence quantification analysis of the logistic equation with transients. Phys. Lett. A 223, 255–260 (1996) Turchin, P.: Complex population dynamics. Princeton University Press, Princeton (2003) Turchin, P., Ellner, S.P.: Modelling time series data. In: Perry, J.N., Smith, R.H., Woiwood, I.P., Morse, D. (eds.) Chaos in Real Data, pp. 33–48. Kluwer, Dordrecht, The Netherlands (2000) Vastano, J.A., Swinney, H.L.: Information transport in spatiotemporal systems. Phys. Rev. Lett. 60, 1773–1776 (1988) Vazquez, D.P., Simberloff, D.: Ecological specialization and susceptibility to disturbance: conjectures and refutations. Am. Natur. 159, 606–623 (2002) Webber, C.L., Jr., Zbilut, J.P.: Dynamical assessment of physiological systems and states using recurrence plot strategies. J. Appl. Physiol. 76, 965–973 (1994) Whitney, H.: Differentiable manifolds. Ann. Math. 37, 645–680 (1936) Wikle, C.K., Royle, J.A.: Space-time dynamic design of environmental monitoring networks. J. Agric. Biol. Environ. Stat. 4, 489–507 (1999) Wikle, C.K., Royle, J.A.: Dynamic design of ecological monitoring networks for non-Gaussian spatio-temporal data. Environmetrics 16, 507–522 (2005) Williams, B.K.: Optimal stochastic control in natural resource management: framework and examples. Ecol. Model. 16, 275–297 (1982) Williams, B.K.: Review of dynamic optimization methods in renewable natural resource management. Natur. Resour. Model. 3, 137–216 (1989) Williams, B.K.: Adaptive optimization and the harvest of biological populations. Math. Biosci. 136, 1–20 (1996) Williams, B.K., Nichols, J.D., Conroy, M.J.: Analysis and Management of Animal Populations. Academic Press, San Diego (2002) Williams, G.P.: Chaos Theory Tamed. Joseph Henry Press, Washington, D.C (1997) Yoccoz, N.G., Nichols, J.D., Boulinier, T.: Monitoring of biological diversity in space and time. Trends Ecol. Evol. 16, 446–453 (2001) Yule, G.U.: On a method of investigating periodicities in disturbed series, with special reference to Wolfes sunspot numbers. Phil. Trans. Roy. Soc. London A 226, 267–298 (1927)

Chapter 9

Markov Entropy Centrality: Chemical, Biological, Crime, and Legislative Networks C.R. Munteanu, J. Dorado, Alejandro Pazos-Sierra, F. Prado-Prado, L.G. Pérez-Montoto, S. Vilar, F.M. Ubeira, A. Sanchez-Gonzaléz, M. Cruz-Monteagudo, S. Arrasate, N. Sotomayor, E. Lete, A. Duardo-Sánchez, A. D´ıaz-López, G. Patlewicz, and H. González-D´ıaz

Abstract In this chapter, we propose the study of multiple systems using node centrality or connectedness information measures derived from a Graph or Complex Network. The information is quantified in terms of the Entropy centrality k C .j / of the j th parts or states (nodes) of a Markov Chain associated with the system, represented by a network graph. The procedure is standard for all systems despite the complexity of the system. First, we define the phenomena to study, ranging from molecular systems composed by single molecules (drug activity, drug toxicity), multiple molecules (networks of chemical reactions), and macromolecules (DNA– drug interaction, protein function), to ecological systems (bacterial co-aggregation), or social systems (criminal causation, legislative productivity). Second, we collect several cases from literature (drugs, chemical reactions, proteins, bacterial species, or criminal cases). Next, we classify the cases in at least two different groups (active/nonactive drugs, enantioselective/non-enantioselective reactions, functional/nonfunctional proteins, co-aggregating/non-co-aggregating bacteria, or crime/noncrime cause, efficient/nonefficient law). After that, we represent the

C.R. Munteanu () Department of Information and Communication Technologies, Computer Science Faculty, University of A Coruña, 15071 A Coruña, Spain e-mail: [email protected]; H. González-D´ıaz () Department of Microbiology and Parasitology, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain Department of Organic Chemistry, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain Department of Inorganic Chemistry, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain Faculty of Pharmacy, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 9, © Springer Science+Business Media, LLC 2011

199

200

C.R. Munteanu et al.

interconnectivity of the discrete parts of the system (atoms, amino acids, reactants, bacteria species, or people) as a graph or network. The Markov Chain theory is used to calculate the entropy of the system for nodes placed at different distances. Finally, we aim to both derive and validate a classification model using the entropy values as input variables and the classification of cases as the output variables. The model is used to predict the probability with which a case presents the studied property. The present work proposes the entropy of a Markov Chain associated with a network or graph to be used as a universal quantity in pattern recognition regardless the chemical, biological, social, or other nature of the systems under study. Keywords Bacteria co-aggregation • Chiral reaction • Complex network • Criminal causation • Drug design • Ecology • Entropy • Graph theory • Markov chain • Organic synthesis • Parasite–host interaction • Political legislative networks • Proteomics MSC2000 Primary 57Q05; Secondary 37B40, 68R10, 65C40, 91D30.

1 General Introduction A network is a system that can be represented for study as a graph with at least two components, the nodes and the edges/arcs. Complex interacting networks are observed in systems from diverse areas such as physics, biology, economics, ecology, and computer science. For example, economic or social interactions often organize themselves in complex network structures. Similar phenomena are observed in traffic flow and in communication networks such as is the Internet. Current problems within the field of biosciences, prominent examples include protein networks in the living cell, as well as molecular networks in the genome. On larger scales, one finds networks of cells as in neural networks, up to the scale of organisms in ecological food webs [27]. In fact, networks are present everywhere. Drug–target interactions, disease–genome correspondences, whole-cell regulation processes, metabolic reactions, protein–protein interactions, sexual relationships, disease transmission, Internet communications, electric power systems, politics, crime, legislative action, scientific collaboration, and many others can be all viewed as network [47, 150]. The elucidation of structural and functional relationships in these and other chemical, biological, technological, and social networks generates the need for a meaningful ranking of network elements (nodes) with numerical indices often known as node centralities. A classic work in this sense was proposed by Sabidussi [184]. Centrality analysis ranks network elements according to their importance within the network structure, and different centrality measures focus on different importance concepts [126]. In the age of information, the discovery or definition of new network systems and the necessity of large database compilation, analysis, and result retrieval necessitate new software and centrality measures. For instance, Centralities in Biological

9 Markov Entropy Centrality: Chemical, Biological, Crime, and Legislative Networks

201

Networks (CentiBiN) is a tool for the computation and exploration of centralities in biological networks such as protein–protein interaction networks. It computes 17 different centralities for directed or undirected networks, ranging from local measures, that is, measures that only consider the direct neighborhood of a network element, to global measures. CentiBiN supports the exploration of the centrality distribution by visualizing central elements within the network and provides several layout mechanisms for the automatic generation of graphical representations of a network. It supports different input formats, especially for biological networks, and the export of the computed centralities to other tools [126]. There are many types of centralities apart from those calculated by CentiBin, but in general we use the symbol Ct (j); where C is centrality, jth refers to a node, and t is substituted for a symbol or acronym for the specific type of centrality (Table 9.1). In addition, there are different relationships between the centralities and topological indices (TIs). TIs are numerical indices that describe the connectedness or connectivity between all nodes in a network and are also very useful for studying global rather than local network properties. For instance, one centrality for a node can be obtained by taking the difference between the TI(G) for the entire graph and the one obtained after removing the jth node [Ct (j) D TI(G) – TI(G-j)]. On the other hand, we can obtain total TIs as the sum of all the node centralities in the graph. Many of these indices have been recently reviewed and discussed in connection with their application to proteomics and medicinal chemistry [81, 83]. Node centralities are numerical parameters that contain information about the connections between a node and the rest of the network. They can be considered local graph TIs. One example is the famous Wiener index (W) used for years in quantitative structure–activity relationship (QSAR) studies of small molecules [210]. The derived centrality called closeness vitality [Cclv D W(G) – W(G-j)] is a classic node centrality often used in complex networks and implemented in CentiBin [126]. Other TIs are the Zagreb group indices, implemented in the Pajek application, dedicated to the complex network analysis and in DRAGON [191], a powerful molecular graph tool. The indices calculated by DRAGON are very useful for small molecule QSAR studies, many of them are summarized in the accompanying handbook [199]. In general, the definition of new TIs or Ct (j) values is still a very active field of research. It is particularly justified in the case of small molecules due to the possibility of obtaining good estimates of biological activity with QSAR models and reducing experimental costs. However, the term QSAR refers to a model that connects the structure of drugs with their biological activity. However, the term could be extended to different systems and properties. In this sense, it could be appropriate to use the acronym QSPR (quantitative structure–property relationships). For this practical reason and others, many authors have introduced new TIs or Ct (j) values. Recently, Estrada has introduced subgraph centrality that characterizes the participation of each node in all subgraphs in a network. Smaller subgraphs are given more weight than larger ones, which makes this measure appropriate for characterizing network motifs. Compared with the number of links per node or node degree centrality Cdeg (j), the ranking introduced by C(S)(i) is

202


Table 9.1 Definitions of some classic Ct (j) and new k-order Markov Chain k Ct (j) node centralities Name Degree Eccentricity Closeness Radiality Centroid values Stress Shortest-path betweenness Current-flow closeness Current-flow betweenness Katz status index Eigenvector Hubbell index Bargaining PageRank HITS authority HITS hubs Closeness vitality Subgraph centrality Markov entropy

Formulaa Cdeg .j / D deg.j / j /g1 Cecc .j / D maxfdist.i; P Cclo .j / D 1= j 2V dist.i; j / P Crad .j / D w2V .G C 1 dist.i; j // = .n 1/ Ccen .j / D min ff .i; j / W i 2 V n fjgg P P Cstr .j / D Ps…v2V Pt…v2V st .j / Cspb .j / D s…j 2V t…j 2V ıst .j / Ccfc .j / D .n 1/= Ccfb .j / D Ckatz D

P s;t2V

P1

kD1

P i…V

pji .j / pij .i /

st .j /=.n 1/.n 2/

˛ k .At /k u

EC.j / D e1 .j / Chubbell D E C W Chubbell Cbrg D ˛ .I ˇA/1 A u Cpagerank D dP Cpagerank C .1 d / u Cauths D AT Chubs Chubs D A Cauths Cclv .j / D W .G/ W .Gn fjg/ Cs .j / D k

P1

lD0

l .j / lŠ

D

PN

iD1

k C .j / D T0 1… k D Pn j D1 k pj log k pj

Œi .j /2 e j

Softwareb CBI CBI

Reference [126] [126]

CBI

[126]

CBI

[126]

CBI CBI CBI

[126] [126] [126]

CBI

[126]

CBI

[126]

CBI

[126]

CBI CBI CBI CBI CBI CBI CBI

[126] [126] [126] [126] [126] [126] [126]

–

[63]

MI

This work

a

All symbols used in these formulae are very common in complex networks theory literature and cannot be explained in detail here. However, G D (V, E) is an undirected or directed, (strong) connected graph with n D jVjvertices; deg(j) denotes the degree of the vertex v in an undirected graph and deg(j)* denotes the valence degree for molecular network only; dist(i, j) denotes the length of a shortest path between the vertices v and w; st denotes the number of shortest paths from s to t and st (j) the number of shortest path from s to t that use the vertex j. D, A, and 1 … are the topological distance, adjacency, and Markov or stochastic matrices of the graph G. For more details, please see the references cited and others b Software used for calculation: CentiBin (CBI) or MARCH-INSIDE (MI)

more highly correlated with the lethality of individual proteins removed from the proteome of S. cereviciae. In this sense, the prediction of protein lethality with a model based on graph indices is an interesting system for network studies and could be considered as a higher analogue of QSPR study in whole proteomes [83]. On the other hand, González-D´ıaz et al. have used Markov Chain Models (MCMs) to describe interesting phenomena inside complex systems. In particular, we used discrete MCMs which associate a stochastic, transition, or Markov matrix (1 …) to the graph or network representation of the systems. In almost all previous


203

works, the authors have mainly used node-to-node adjacency matrices (A), node-tonode topological distance matrices (D), or edge-to-edge adjacency matrices (E). The Markov matrices were selected by our group because they give a simple but realistic interpretation of the system in terms of the probability of interaction or communication between the parts of the system (nodes or states) placed at different distances. The method was initially named MARCH-INSIDE (Markov Chemicals In Silico Design) focusing on applications to small molecules. Finally, we retained the acronym but spelt it out in a different way MARCH-INSIDE (Markov Chain Invariants for Network Simulation and Design) making reference to applications on more general classes of systems. The MARCH-INSIDE approach use different classes of TIs and C (j) values to describe networks that are obtained in the form of invariants of the stochastic matrix 1 …. These indices include local node indices of the following families or classes: spectral moments, mean values, absolute probabilities, and entropy information indices. Specifically, node Markov entropy indices (symbolized here as C (j)) have been very useful to describe both local or total properties of many systems. The authors note three recent reviews discussing the applications ranging from graph of small molecules to graph or network representation of protein sequences and 3D structure, DNA sequences, RNA secondary structure, or human blood proteome mass spectroscopy outcomes [81, 83, 93]. In conclusion, in all these works, including MARCH-INSIDE and the other models, we noted that different TIs or Ct (j) values can be used at different structural levels, and interestingly the same indices of graphs or networks of one level can be used as inputs of QSPR models to predict graphs or networks of higher levels. The present chapter is dedicated to the applications of the C (j) values. Thus, we shall develop nine case studies that test the universal use of the Markov entropy centralities k C (j) to recognize structural patterns and/or predict properties of systems, ranging in very different large scales. Experiments 1–3 lie within the scope of small molecules chemistry, experiment 4 is devoted to proteins, and experiments 5 and 6 to microorganisms. Finally, the experiments 7 and 8 lie within a region that does not contain any type of molecular information, but information related to social phenomena. In detail, these experiments may be described as follows: • Experiment 1: Prediction of the chemical skin sensitization with k C (j) values of molecular graphs. • Experiment 2: Introduction of a QSAR model to predict the action of small molecules over multiple targets and construct drug–target networks based on weighted k C (j) values of the molecular graph. • Experiment 3: Chemical reaction study using the k C (j) values of different molecules. • Experiment 4: Prediction of the protein thermal stability and protein anticancer action by DNA-cleavage mechanism with the k C (j) measures of protein structural networks. • Experiment 5: Calculation of scoring functions for DNA–drug docking using the k C (j) of lattice network representations of molecular dynamic trajectories.

204


• Experiment 6: Prediction of the bacterial co-aggregation and reconstruction of the bacteria–bacteria co-aggregation networks with the k C (j) of phylogenetic tree graphs. • Experiment 7: Evaluation of the host–parasite interaction networks using the k C (j) of the phylogenetic co-evolution graphs. • Experiment 8: Reports a model to discriminate the main from the secondary causes or actions based on k C (j) values of the path that connects cause and consequence in Criminal networks. • Experiment 9: Focuses on k C (j) values of legislative production networks of the Spanish tributary system.

2 Experiments 2.1 Experiment 1 2.1.1 Introduction to QSTR Studies Environmental pollution causes many occupational health problems. Allergic contact dermatitis is considered one of the most frequent health problems [68]. For many years, the species used for the identification of the contact allergens was the guinea pig. A variety of guinea pig test methods have been described, including the guinea pig maximization test, the mouse ear swelling test, Buehler occluded patch test, and so on [130]. It is clear that such assays are subject to some significant limitations. These limitations have encouraged the consideration of alternative approaches such as the local lymph node assay (LLNA) [16]. In summary, the LLNA provides a novel approach to the identification of skin allergens where immunobiological events are measured. The LLNA requires fewer animals than needed for a standard guinea pig maximization test and provides a quantitative measure of sensitizing potential and potency [68,93,184]. However, both the guinea pig tests and the LLNA are in vivo methods and, as a consequence, will not eliminate the use of animals in the assessment of contact sensitizing activity [130]. In this sense, QSAR or possibly more precisely quantitative structure–toxicity relationships (QSTR) studies are already used implicitly to reduce animal testing through the rational design of new chemicals for different purposes [167]. In any case, despite the potentialities to solve the present problem, we have never studied LLNA results with the k C (j) values. In this connection, in the two next sections, we give the theoretical basis of the k C (j) method adapted to molecular graphs and we explore its potential to fit LLNA for the first time. 2.1.2 Markov Entropy Node Centrality k C .j / for Molecular Graphs In addition to the classical information measure such as entropy for characterizing graphs [23], Dehmer [49, 50] proposed a series of information-theoretic concepts


205

for the analysis of the complex networks by determining the structural information content of graphs. For example, [50] provides an approach that is based on tree decompositions. In the actual chapter, we used the Markov Chain theory and Shannon’s equation to calculate the entropy node centrality values k C (j) associated with electronic distribution in a molecule. The values k C (j) are referred to atoms as nodes in molecular graphs. First, we have to calculate the absolute probabilities pk (j) for the distribution of electrons on the atom j. These values can be determined as the elements of the vectors k . These vectors are elements of a Markov chain based on the stochastic matrix 1 …, which contain elements that describe the probabilities of transition of electrons p1 (i,j) from node (atom) ith to jth as following: k k s D 0 .s/ 1 … .s/ D Œs p0 .1/; s p0 .2/; s p0 .3/; :s p0 .n/ 2

3k p1 .1; 1/ s p1 .1; 2/ : : s p1 .1; n/ 6 s p .2; 1/ s p .s/ : : 7 : 6 1 7 1 6 7 6 : : :: : 7 : 6 7 4 5 : : :: : s p1 .n; 1/ : : : s p1 .n; n/ s

(9.1)

The theoretic foundations of the method have been given in previous works; hence we do not detail it here but refer the reader to these work [84]. After that, the entropy centrality measures k C (j) can be very easily calculated by applying the Shannon’s formula to each element pk(j) of the vectors (s). The k C (j) values can be summed for specific atom sets (AS) or for the same groups of nodes to create local molecular descriptors; however, if we sum all atom centrality in the molecule, we obtain a total molecular descriptors. Herein the AS used were: halogens (X), unsaturated carbons (Cins ), saturated carbons (Csat ), heteroatoms (Het), hydrogens bound to heteroatoms (H-Het), and all atoms (T). The corresponding symbols of the local entropy centrality for these AS are: k C (X), k C (Cins ), k C (Csat ), k C (Het), k C (H-Het), and k C (T). When we refer to k C (T) for all atoms, we shall omit the T and we shall write simply k C from now on. In this study, we calculated the first six classes of entropy centrality (k D 0 to 5) for the 5 AS, in total 30 (6 5) molecular local centralities for each compound. In the following, we give the formula for both the transition probabilities (elements of the matrix) and the atoms set entropy centrality measures. Since none of the chemicals in the LLNA set were drugs, suggest modifying drug to compound or equivalent. Many of the chemicals are those used in personal care products – including fragrances. s

j p0 .j / D Pn

kD1

k

;

(9.2)

ıij j p1 .i; j / D Pn ; kD1 ıik k X X k s k C .AS / D C .j / D pk .j / logŒs pk .j /: s

j 2AS

j 2AS

(9.3) (9.4)

206


2.1.3 Data Analysis, Results, and Discussion QSAR techniques are perhaps most widely used in the pharmaceutical industry. The other great contribution of QSAR/QSTR approaches to the reduction and refinement of animal testing is in screening out potentially toxic chemicals before their synthesis or manufacture, and in prioritizing drugs for biological assay. The same observations apply to entropy-based QSTR models [84]. In this context, we continue here with a preliminary exploration of the potential of k C (j) values to classify organic substances as potential contact allergens (or not) as measured by murine LLNA assay [68]. The best equation found was the following: LLNA% D 926:2 2 C 354:8 3 C 4514:9 4 C C 3793:4 5 C C1619:9 6 C 1472:1 7 C C 6:37 N D 97

U D 0:4

p < 0:001

(9.5)

LLNA%, the output of the model, represents a continuous variable (not probability) and represents the predicted scores of toxicity according to LLNA test. The statistic parameters of this equation are: the number of cases (N) and the Wilks’ statistic (U), which for overall discrimination takes values in the range from 0 (perfect discrimination) to 1 (no discrimination). The Fisher ratio (F) allowed us to confirm the hypothesis of separation of groups with a probability of error (p-level) p < 0.05. This model presents an accuracy of 90.72% of the compounds in the training series, i.e., 9 misclassifications out of 97 cases, while in the predicting series there were 4 errors in 34 cases, i.e., a level of 88.23% of accuracy. More specifically, the model correctly classifies 90.14% of contact allergen compounds in the training series (64 of 71 chemicals) and 85.7%, i.e. 2 misclassifications out of 14 cases of these compounds in the predicting ones. In Table 9.2, we illustrate the results obtained for some compounds.

2.2 Experiment 2 2.2.1 Introduction to Multi-Target QSAR of Antimicrobial Drugs The discovery of antibiotics is perhaps the most significant therapeutic advance in medical history. In particular, infections caused by bacteria have increased dramatically during the past years [183]. Water purification, immunization (vaccination), and modern antibiotic treatment continue to reduce the morbidity and the mortality of bacterial disease in the twenty-first century, at least in the developed world where these are acceptable cultural practices. However, many new bacterial pathogens have been recognized in the past 25 years, and many bacterial pathogens, such as Staphylococcus aureus and Streptococcus pneumoniae, have emerged with new forms of virulence and new patterns of resistance to antimicrobial agents [144].

9 Markov Entropy Centrality: Chemical, Biological, Crime, and Legislative Networks Table 9.2 Name and predicted posteriori probabilities for some LLNA allergens Name P%a Name 12-Bromododecanoic acid 97.48 Dodecylmethane sulfonate 12-Bromodecanoic acid 97.49 Ethylenglycolmetacrilate 2-Bromotetradecanoic acid 91.20 Ethylendiamine 2-Chloromethyl fluorene 96.95 Aldehyde hexilcynnamic 2-Hydroxy ethyl acrylate 90.83 Iodo hexane 2,4,5-Trichlorophenol 59.08 Iodo nonane 2,4,6-Ttrichloro-1,3,5-triazine 99.88 Isoeugenol 2-(N -acetoxy-acetamido)fluorene 70.49 Benzenesulfonate 3-Bromoethyl-3-dimethylfuranone 99.96 Methyl-2-sulfonoethyloctadecane 3-Phenylbenzoate 24.94 Dodecilmethanesulfonate 3,4-Dihydrocoumarin 45.61 Methylhexadecane sulfonate 3,5,5-Trimethyl-chlorohexane 98.19 Methyl isoeugenol 3-Methyleugenol 21.87 Methyl-N -nitroso urea 4-Nitrobromobenzyl 75.40 Octylgallate 4-Nitrochlorobenzyl 32.96 Oxalone 5-Methyl-eugenol 44.81 Penicillin G 7-Bromo tetradecane 93.20 Quinol 7,12-Dimethyl-1-benz(a)antracene 90.35 1-Bromo hexadecane Cinnamic aldehyde 21.78 Lauryl sulfate Benzo[a]pyrane 11.88 Tetramethyl thiuramdisulfide Benzoquinone 15.74 Tioglicerol Butilglycidyl ether 97.39 Amonio tioglycollate Camphorquinone 94.62 Vinilpiridina Amonio tioglicollate 99.22 Pyridine Chlorpromacina-e 82.79 1-Ethyl nitroguanidine Chlorohexadecane 91.02 Citral Chlorooctadecane 91.05 Sultone Diethylentriamine 55.60 Naphtoflavone1 Diethyl sulfate 97.38 Naphtoflavone 2 a

207

P%a 91.31 99.86 37.90 92.04 91.97 91.40 77.31 13.50 92.79 91.28 55.57 97.34 27.82 92.91 96.28 3.82 87.40 97.85 91.59 92.18 81.87 98.12 95.49 73.72 15.83 99.65 53.42 73.64 51.03

Difference between probability of skin sensitization and probability of non-skin sensitization

At risk are patients after organ transplantation treated with immunosuppressives or those suffering with a weakened immune system, for example, patients with AIDS. However, the speed of discovery and marketing of new antimicrobial drugs, including antibacterial, in general has slowed down, and there is growing appreciation that much of the low-lying fruit accessible to traditional methods of antimicrobial discovery has been harvested. Combating emerging drug-resistant strains of infectious agents may require the adoption of fresh approaches to drug design and target discovery [159]. Using computational models to reach these goals may increase the rational use of sources in the discovery process. In general, there are many variations to the QSAR drug discovery strategies, but all of them follow more or less a series of common stages. A summary of the steps often performed in QSAR-based drug discovery has been published before [14, 73]. However, inspection of these steps clearly shows that the collection of biological activity data, chemical structure codification, and data analysis are three of the most

208


important bottlenecks in QSAR-based drug or target discovery. The codification of chemical structure information with different TIs has been a very active field of research. In fact, the QSAR studies may play an important role being used as predictive tools for the molecular development [36]. Unfortunately, the QSAR studies are generally based on databases considering only structurally parent compounds acting against one single microbial species. Up until today, there are nearly 1,600 molecular descriptors that theoretically may be generalized and used to solve the former problem [199]. Many of these indices are TIs or simply invariants of a molecular graph. In another review, our group have discussed recent advances in the field [66]. However, in spite of its great potential, in general, TIs or local TIs (sum of node centralities Ct (j)) and other indices have not been extended to allow multi-tasking or multi-target (mt) prediction of biological properties [61]. We shall call this alternative as an input-coded multi-tasking learning approach, and by extension we have input-coded mt-QSAR models. The method is very flexible and can be extended to any type of molecular indices (TIs included) [149]. Multi-tasking QSAR (mt-QSAR) [61] can be defined as the prediction of multiple outputs with a single model, and it is closely related to the more general term multitasking learning (used in cognitive sciences) [149]. It means that we can predict, for instance, several mechanisms of actions, partition coefficient in different biphasic systems, inhibition of different cancer lines, or activity against different microbial species to any drug using a single model. We decided to introduce information relative to the type of property to be predicted inside the molecular descriptors. In our mt-QSAR approach, we have different value of the TIs or local TIs (the sum of graph centrality values Ct (j)) for the same molecule depending on, for example, the specific biphasic system in which we want to estimate the partition coefficient of the drug [41] or the specific drug side effect we want to predict. Anyhow, despite the importance of this problem, we have not found a previous application of k C (j) values to resolve it. In particular, the use of QSAR models to predict the biological activity of antimicrobial drugs is being largely investigated. In a recent review [93], we illustrated the results of a search in the SciFinder platform, using the term QSAR combined with the four main groups of antimicrobial drugs (antibacterials, antiviral, antifungic, and antiparasitic drugs). The search shows a clearly increasing trend in the uses of different antimicrobials discovery QSAR methods in the last years, but almost all of them are not mt-QSAR models and consequently are restricted to a narrow search on limited series of compounds and/or targets. In the same review [93], we discussed the several applications of MARCH-INSIDE to the Medicinal Chemistry of Antimicrobial agents as well as their molecular targets. First, we revised the use of classic TIs for prediction of the antiparasitic compounds for the treatment of Fascioliasis. Next, we revised the use of chiral TIs (CTIs) to predict new antibacterial, antiviral, and anti-coccidial compounds. After that, we review multitarget TIs (mt-TIs) with unify QSAR models predicting antifungal, antibacterial, or antiparasitic drugs with multiple targets (microbial species). We also discussed the uses of mt-TIs to assemble drug–drug similarity complex networks of antimicrobial compounds based on molecular structure.


209

With the mt-QSAR generalization of QSAR models (including but not limited to MACRH-INSIDE), we selected pairs of antiparasitic drugs with similar/dissimilar predicted multi-species activities and represented them as a complex network (CN). We call this type of CN the drug–drug multi-species CN (msCN). Note: do not confuse the network used to represent the molecular structure of the drug (molecular graph) and the network of drug–target interactions. The first refers to one molecule only (the nodes are atoms and the edges chemical bonds). Conversely, the second refers to many drugs and targets such that each node is a drug, target, or drug–target pair, and the edges express relationships between pairs of drugs and/or targets [213]. In fact, we can use the first network (molecular graph) or the TIs of this network as inputs to predict which pairs of nodes (drugs or targets) are connected in the second or output network (here the msCN) [212]. The msCN is useful, for instance, to identify drugs with similar mechanism of action or similar activity against many different species. At the same time, we may invert the procedure and selected pairs of parasites species with similar/dissimilar drugs sensibility to construct a parasite– parasite multidrug resistance CN (mdrCN) [93,173,174]. The mdrCNs may be used to identify parasites species with sensibility for the same drugs to select parasites with specific resistance to drugs. In any case, the MARCH-INSIDE approach is also able to calculate entropy centralities such as the k C (j) values we are studying here. Anyhow, we did not reported a mt-QSAR study based on k C (j) values before. Consequently, in the next two sections, we give the theoretical basis and discuss the results obtained when we extend, by the first time, k C (j) values to calculate k C ,s(j) centralities useful to perform mt-QSAR for different antibacterial species(s). 2.2.2 Multi-Target Markov Entropy Node Centrality k C;s .j / for Molecular Graphs Using Chapman–Kolgomorov equations, we can calculate multi-target k C ,s(j) values referred to atoms (nodes) in molecular graphs. As was mentioned above multi-target here means that we obtain different k C ,s(j) values for the same atom in the same molecule when the molecular target (bacteria, virus, parasite, receptor, enzyme, etc.) change. First, we have to calculate the absolute probabilities s pk (j) for the interaction in many step of different jth atoms with the specific target. Here targets are only different microbial species (s). In this sense, we insert the superscript “s” in the symbol of the centrality. These values can be determined as the elements of the vectors k (s). These vectors are elements of a Markov chain based on the stochastic matrix 1 …, which describes probabilities of interaction s p1 (i,j) of the jth atom given that previously other ith atom has interacted with the target (9.1)–(9.4). The specificity for one target is given using target-specific weights in the definition of the elements of the matrix 1 …. The theoretic foundations of the method have been given in previous works; hence we do not detail it here but refer the reader to these works [94, 172]. After that, the entropy centrality is very easy to calculate applying the Shannon’s formula to each element s pk (j) of the vectors k (s)and obtain the entropy centrality measures k C ,s(j). As in the example 1, we can sum the

210


k

C ,s(j) values for specific atom sets (AS), or the same groups of nodes, to create local molecular descriptors for the drug–target interaction. Herein the AS used were: halogens (X), insaturated carbons (Cins ), saturated carbons (Csat ), heteroatoms (Het), and hydrogens bound to heteroatoms (H-Het). The corresponding symbols of the local entropy centrality for these AS are: k C ,s(X), k C ,s(Cins ), k C ,s(Csat ), k C ,s(Het), k C ,s(H-Het), and k C ,s(T). In this study, we calculated the first six classes of entropy centrality (k D 0–5) for the 5 AS in total 30 (6 5) molecular local centralities for each drug [172].

2.2.3 Data Analysis, Results, and Discussion One of the main advantages of the present approach is that the generalized parameters k C ,s(j) fit on more large and complex databases than the previous ones. This work introduces for the first time a single linear mt-QSAR equation model to predict the antibacterial activity of drugs against different species. The data set used here was established by a set of marketed and/or very recently reported antibacterial drugs with low reported MIC50 < 10 M against different bacterial strains. The data set was formed by 80 different drugs experimentally tested against some species of a list of more than 90 bacteria. Not all drugs were tested in the literature against all listed species; hence we were able to collect 1,200 cases (drug/species pairs) instead of 7,200 (80 90) cases. The names or codes and activity for all compounds as well as the references used to collect it are depicted in a supplementary material file, available upon author request. DB score D 3:95 0 C;s .Csat / C 2:65 0 C;s .H Het/ C 2:86 5 C;s .Cins / C1:53 5 C;s .T / 2:03 1 C;s .H Het/ 5:76 1 C;s .Total/ 1:70 U D 0:48

F D 94:767

p < 0:001

(9.6)

DB-score, the output of the model (thresholdD0), is a real value variable (not probability) that scores drug–bacteria activity specificity. In this equation, k C ,s(j) were summed for the totality (T) of the atoms in the molecule or for specific atom sets (AS) as we referred above. These collections are atoms with a common characteristic as for instance are: saturated carbon atoms (Csat ), hydrogen atoms linked to the hetero-atoms (H-Het). The model correctly classifies 829 of 950 active compounds (87.30%) and 203 of 245 non-active compounds (83.06%). Overall training accuracy was 85.18%. Validation of the model was carried out by means of Leave-Species-Out (LSO) procedure (see Table 9.3). After elimination step-bystep of all drugs tested against one specific species, we recorded the percentage of good classification of leave-out compounds (LSO predictability). In addition, the robustness of the model to the elimination of the compounds (LSO-robustness) was considered. This aspect was considered as the variation of the percentage of

9 Markov Entropy Centrality: Chemical, Biological, Crime, and Legislative Networks Table 9.3 Results for mt-QSAR model Bacteria species Drugsa Bacteroides thetaiotaomicron 9 Bacillus spp. 11 Bacteroides distasonis 10 Bacteroides fragilis 17 Bacteroides ovatus 11 Bifidobacterium spp. 13 Clostridium spp. 19 Corynebacterium spp. 11 Escherichia coli 30 Enterococcus faecalis 37 Enterococcus faecium 29 Enterococcus spp. 14 Fusobacterium nucleatum 15 Fusobacterium spp. 13 Haemophilus influenzae 13 Klebsiella pneumoniae 17 Klebsiella spp. 13 Lactobacillus spp. 14 Moraxella catarrhalis 16 Pseudomonas aeruginosa 13 Peptostreptococcus spp. 23 Prevotella intermedia 14 Prevotella melaninogenica 14 Propionibacterium acnes 18 Rhodococcus equi 14 Streptococcus pneumoniae 24 Staphylococcus aureus 32 Staphylococcus epidermidis 22 Average

17

Observedb 88.9 100 90 88.2 90.9 92.3 89.5 100 96.7 97.3 96.6 100 93.3 92.3 92.3 94.1 100 92.8 100 100 95.6 92.8 100 88.9 100 100 96.9 100 95.34

LSOc 86.93 86.76 86.97 87.07 87.05 87.02 86.89 86.77 86.80 86.62 86.67 86.89 87.01 86.84 86.93 86.82 86.81 86.93 86.87 86.88 86.95 87.02 86.94 86.75 86.85 86.63 86.80 86.95

d 3.98 3.80 4.02 4.12 4.09 4.07 3.93 3.82 3.85 3.66 3.72 3.94 4.05 3.88 3.97 3.87 3.85 3.97 3.92 3.92 4.00 4.07 3.99 3.80 3.90 3.68 3.85 4.00

86.87

3.92

211

a

Number of drugs tested by species Observed drugs active against this species c Predicted drugs active against this species after leave-species-out validation (LSO) d LSO robustness () b

good classification of the modified model () in LSO with respect to the original one. Average LSO predictability was 86.74 ˙ 0.12% (average ˙ SD) and D 3.92%, being 17 the average number of drugs tested against each bacteria species. Results for some of the 140 studied species were Bacteroides thetaaiotaomicron: 9 tested compounds, 88.9% of LSO predictability, D 3.98%; Escherichia coli 30, 96.7%, D 3.85%I Enterococcus faecium 29, 96.6%, D 3.72%I Klebsiella pneumoniae 17, 94.1%, D 3.87%I Staphylococcus aureus 32, 96.9%, D 3.85%I Streptococcus pneumoniae 24, 100%, D 3.68%. Next, we used the outputs of the mt-QSAR as inputs to construct the first CN for antibacterial drugs and species based on k C ,s(j) values. In previous works, we

212


constructed for the first time mt-QSAR models accounting for pairs of antiparasitic [93, 173], antifungal [92, 94], or antiviral drugs [174] with similar/dissimilar multispecies activity profile and represented it as large networks. In this work, we have had to contend with a very high number of possible drug–bacteria pairs (DBPs). These DBPs may be investigated using CNs to regroup or cluster drugs with similar multi-bacterial affinity profile. In DBP-CN, the DBPs are nodes interconnected by the edges if they have similar drug–bacteria activity. We need to measure the activity of the drug on different bacterial if we cannot predict it. We propose to construct here, by the first time, a DBP-CN taking into consideration only the number of DBPs predicted by the mt-QSAR model based on k C (j) values. In order to construct this CN, we have given the following steps: 1. First, we calculated two types of activity Z-scores (drug score and bacteria score) for both experimental and QSAR-predicted values: zobs .d / D

log MICi log MICmax

zpred .d / D p./i ;

(9.7) (9.8)

where d is the score affinity, either observed score (sobs ) or predicted score (spred ). sobs was calculated on the experimental data (IC50 ). We calculated the spred of each one of the 1,031 drugs with all the studied bacteria here by substituting the molecular descriptors into the QSAR equation using the Microsoft Excel application [153]. Mean is the average of either sobs or spred for the DBP. We calculate the distance matrix between all DBP using a Euclidean distance: obs

pred

dij D

1 j log MICi log MICj j log MICmax

dij D jp./i p./j j

(9.9) (9.10)

2. Using Microsoft Excel [153] again, we transformed the DBPs distance matrices derived into Boolean matrices. The elements of this type of matrix are equal to 1 if two DBPs have an Euclidean distance dij < cutoff value. We explore the threshold values in a range from log MICsobs until MICspred trying to obtain average DBP node degree equal to 1 and minimizing the number of disconnected DBPs. The line command used in Excel to transform the distance matrix into a Boolean matrix was f D if($N11 D P$9,0,if(ABS($O11-P$10)0) in the molecular backbone. In order to reach this goal, the method uses an MCM, which determines the absolute probabilities k m p( j ) with which the atoms placed at different distances k in the molecule (m) affect the contribution of the atom j to the molecular property in question. These k p(m j ) probabilities can be obtained using the MCM as the elements of a vector k .

216


2 k

D

D

0

k

m

p . 1 / ;

0

p .m 1 / ;

k

6 6 6 : : : p . n / 6 6 4

m

0

p . 2 / ;

m

: : : k p .m n / :

p .m 2 / ;

1

p1;2 1 p2;1 : : 1 pn;1

1

p1;2 1 p2;2 : : :

1

p1;3 1 p2;3 : : :

: : : : :

1

p1;n 1 p2;n : : 1 pn ;n

3k 7 7 7 7 7 5

(9.12)

The matrix in the center of the equation is the so-called stochastic matrix (1 …). The values of this matrix (1 pij ) are the probabilities with which every atom affects the parameters of the atom bonded to it. Both kinds of probabilities k p(j m ) and 1 pij are easily calculated from atomic parameters (j ) and chemical bonding information: 0

m

p.m j / D Pn

j

kD1

1

m

k

;

ıij m j : pij D Pn m kD1 ıik . k /

(9.13) (9.14)

The difference is that in the probabilities 0 p(j m ), we consider isolated atoms by carrying out the sum in the denominator over all n atoms in the molecule. On the other hand, for 1 pij chemical bonding is taken into consideration by means of the factor ıij . This factor has the value 1 if atoms i and j are chemically bonded and it is 0 otherwise. All calculations were performed using the program MARCHINSIDE version 3.0 [89] that can be obtained for free academic use, upon request, from the corresponding author of the present work. In addition, we transformed all nonstructural reaction r parameters such as addition temperature Ta , reaction temperature Tr , and reaction time tr into an entropy scale. The transformation was necessary to express all information (structural and non-structural) into the same entropy scale. For it, we normalized the values into a 0-to-1 scale and obtained the product of this value for the logarithm of the value. The new parameters should be considered as non-structural entropies, encoding important information about reactions. This additional non-molecular entropies are represented as follows: ( r ). Specifically we have: (Ta ), (Tr ), (tr ). . . , etc. 2.3.3 Data Analysis, Results, and Discussion Given the structural or molecular entropy centrality values C (m ) and the nonstructural or reaction entropy values ( r ), we can calculate the following differences C (m ) D C (m ) – C (m ) and ( ) D ( r2 )- ( r1) for any pair of reactions r1 and r2. Using these differences between pairs of reactions C (m ) and ( ) as inputs, we performed a multiple linear regression (MLR) analysis to fit the QSRR equation with the form: X X ee.R/%pred D bm k C .m / C b r . r / C b0 : (9.15) s;l;o;p

r


217

The parameter ee(R)%pred predicts the difference in enantiomeric excess for R-product between two pair of reactions. In this equation, b represents the coefficients of the variables in the model determined with MLR module of the software package STATISTICA 6.0 [193]. We used Forward Stepwise algorithm for the variable selection. The data set composed a set of reported organolithuim addition to imines in the presence of chiral ligands reactions. In the supplementary material, available upon request to corresponding author, we depict all the parameters necessary to evaluate a reaction with the QSRR as well as the data set used, including corresponding structures, in the form of SMILE codes, for all compounds involved in this study. The best model found using the reaction-to-reaction differences on molecular entropy C (m ) and non-molecular entropies ( ) was the following: DEE score D 0:83 C 180:43 C Hp 135:04 C Mp 14:79 C Mleq C 8:42 C .Pl / C 7:43 C .Ml / C11:18 C .So / 11:10 C .Mo / 9:15 C .Hs /

C6:55C .Ps / 7:27 .tr / 12:71 .Tr / C 9:15 Ay 2 n D 17404 R 2 D 0:823 Radjusted D 0:823 F D 6731:5 p < 0:001: (9.16)

DEE-score, the output of the model, is a real value variable (not a probability) that scores the possibility of existence of a Difference of Enantio-Excess (DEE) between two reactions. n is the number of cases (reaction pairs) used to train the model, R2 and R2adjusted are the train and adjusted square regression coefficients, F is Fisher ratio, and p the level of error. All these reactions were previously reported in the literature [8, 12, 30, 31, 51, 54, 76, 114, 118, 119, 132, 170, 196]. The more important molecular variables were the differences in Markov entropy centrality between the initial and final reaction for: • C (Hp) entropy difference on hardness of products • C (Mp) entropy difference on product molar refractivity • C (Mleq) entropy difference on chiral ligand molar refractivity multiply by equivalents • C (Pl) entropy difference on chiral ligand partition coefficient • C (Ml) entropy difference on chiral ligand molar refractivity • C (So) entropy difference on organolithium steric constant • C (Mo) entropy difference on organolithium molar refractivity • C (Hs) entropy difference on substrate hardness • C (Ps) entropy difference on substrate partition coefficient Other non-molecular variables in entropy scale were also selected such as nonmolecular information related to the reaction time (tr ), reaction temperature (Tr ), and average yield for reactions using same procedure (Ay ). This model, with 12 variables, predicts correctly 90.7% of variance of the data set with a standard error of 27.88%. Notably, the values of R2 and R2adjusted are equal, which

218


Fig. 9.1 Observed vs. predicted values of enatioselectivity change for 17404 reaction pairs predicted with the k C (j) values calculated with the molecular graphs of different reagents and products

indicates that the model is not over-fitted due to incorporating an elevated number of parameters. In Fig. 9.1, we plot the observed ee(R) % values vs. the values predicted with the model. In order to validate the model, we used it to predict 26,106 reactions pairs never used to train the model (validation series). In this series, the results were: R2 D 82.04%, F D 9935.9, and p < 0.00001. The model correctly explains 90.5% of variance of the data set with a standard error of 28.22% in the validation series. These results indicate that we developed an accurate model according to previous reports on the use of MLR in QSRR [35, 171, 179]. Using this model, we can construct QSRR-based charts to visually depict the influence of the change in reaction variables over the enantioselectivity of the reaction [95]. This kind of analysis, known as desirability analysis (DA), allows us to predict which levels of the reaction variables ensure a desired enantioselectivity [41]. It could be used to optimize the reaction by changing only one property by organic synthesis modification of substrate or chiral ligand or modifying a reaction condition. As a conclusion of this experiment, we have shown that using the MARCH-INSIDE approach, it is possible to obtain an entropy-based MLRQSRR classifier that can predict the probability of chirality inversion of amines, synthesized by the addition of organolithium reagents to imines, in the presence of the chiral ligands. The model based on entropy study can be used as a tool for preliminary screening of reactions without relying upon geometrical optimization of substrate, organolithium, chiral ligand, or product structure.


219

2.4 Experiment 4 2.4.1 Introduction to Protein QSAR In the post genomic era, there is an increasing need for quick computational chemistry methods that can accurately predict the properties of proteins. In this sense, QSAR is a widely covered field, with many descriptors introduced up to now [56, 127, 199], but most being applied only to small molecules. Nevertheless, the QSAR studies for protein 3D structures may be considered as an emerging field [4,13,116,177]. One of the most promising applications of QSAR to proteins could relate to the prediction of thermal stability, which is an essential issue in protein science [11, 88, 154, 215]. However, in these years we see growing considerably the number of proteins with a defined 3D structure but with unknown biological function. In this sense, a lot of researcher groups indented, through computational methods, to study not only the stability but also the relationship between the structure and the function of the proteins. The methods applied in this field are various like the work published by Dobson and Doig (D&D), where they have shown the possibility to predict proteins as enzyme or not, from the spatial structure, without resorting to alignments. In the cited paper, the authors used 52 protein features and a nonlinear support vector machine (SVM) to classify more than 1,000 proteins with 77% accuracy [57]. In addition to the work of D&D, we found more SVM models that are able to distinguish between an enzymatic and a nonenzymatic protein. Most of them have an overall accuracy lower than our model with an overall accuracy of around 70% based also on topological indices [160]. In this sense, a group of researchers have recently published a review on the growing importance of machine learning methods in the prediction of protein functional class, independently of the sequence similarity [111]. In our opinion, the 3D-QSAR methodology, usually applied to small-sized molecules, may become an alternative to sequence methods for 3D-structure–function predictions in proteins. The ability to predict protein function from the 3D structure experimentally determined is becoming increasingly important as the number of structures resolved is growing more rapidly than our capacity to study the function [102]. Among all molecular descriptors, the 3D indices (3DIs) give a more detailed description of protein structure, while TIs [199] serve for less precise but fast calculations. 3DIs can be calculated from the 3D X-ray crystal structures or from the NMR-fitted protein model [62,67], whereas TIs are graph invariants of different kinds of proteins or DNA graph representations. Here we selected spatial descriptors known as 3D contact map network entropies as 3DIs since they have been widely used to describe both 3D and 2D structural features in different contexts such as polymer sciences, solid phase chemistry, and theoretic chemistry [78–80, 156, 169]. Entropy theory has been routinely applied in many applications such as encoding structure governing biological activity [1, 46, 129], distinguishing between natural product and synthetic molecules [192], computational design of proteins [189], or

220


to detect protein homology [7], but it has never been used in 3D QSAR to develop models capable of distinguishing between enzyme and nonenzyme forms where only the 3D structure of proteins was available. 2.4.2 Markov Entropy Centrality k C .j / for Amino Acids in Protein 3D Structure Networks Here we give general MCM formula for the calculation of the 3D-entropy centrality measures k C (j) of the electrostatic interactions around the jth amino acid, considering neighbor amino acids at different distance k. The method uses the stochastic matrices of electrostatic interaction probability 1 …f and the probability vector of electrostatic potential for isolated amino acids e 0 . With these objects, we can determine the vector e `k of the entropy for electrostatic interactions of amino acids placed at distance k with the jth amino acid, which elements are the k C (j)e centrality values. The general formula to calculate k C (j) measures is as follows: k

C .R/ D

X j D12R

e

k

C .j / D

n X

k

pj .e/ log

k

pj .e/

(9.17)

j D12R

k D e T0 e …k e 0 D e T0 .e …1 /k e 0 :

(9.18)

The superscript e points to the type of molecular force field used (electrostatic). The stochastic matrices are built up as squared matrices (n n), where n is the number of amino acids (aa) in the protein. The stochastic matrices used may encode any potential field. In order to extend the method, we can consider a hypothetical situation in which every jth-aa has general potential fj at an arbitrary initial time (t0 ). All these potentials can be listed as elements of the vector 0 'f . It can be supposed that, after this initial situation, all the aa interact with every other aaj in the protein with interaction energy 1 Eij . For the sake of simplicity, a truncation function ˛ij is applied in such a way that a short-term interaction takes place in a first approximation only between neighboring aa (˛ij D 1). Otherwise, the interaction is forbidden (˛ij D 0). Neglecting direct interactions between distant aa in 1 …f does not avoid the possibility that potential interactions propagate between those aa within the protein backbone in an indirect manner. Consequently, in the present model long-range interactions are possible (not forbidden) but estimated indirectly using the natural powers of 1 …f . The use of MCM theory allows a simple and fast model to calculate the average values of (k ) considering indirect interaction between an aaj and another aai after previous interaction of aaj with other k neighbor amino acids. It is remarkable that the average general potentials k (f ) depend on the absolute probabilities A pk (j) with which the amino acids interact with other amino acids and their k-order. The potential k also depends on the initial unperturbed potential of the amino acid. In the equations represented above, the Apk(j) values are calculated with the vector of absolute initial probabilities, 0 f , and the matrix 1 f based on the Chapman–Kolgomorov equations. In order to carry


221

out the calculations referred in the former equations for any kind of potential adapted here for the electrostatic potential, the elements (1 pij ) of 1 …f and the absolute initial probabilities A pk (j) were calculated as follows: 1

˛ij Eij ˛ij j .wi ; wj ; dij / ˛ij 'j pij D PıC1 D PıC1 D PıC1 mD1 ˛im Eim mD1 ˛im j .wi ; wj ; dij / mD1 ˛im 'm (9.19)

1

qi qj dij

˛ij ˛ij Eij pij D PıC1 D PıC1 mD1 ˛im Eim mD1 ˛im

qi qm dim

D

qj dij

˛ij qi PıC1 qi mD1 ˛im

qm dim

q

˛ij djij ˛ij 'j D PıC1 D PıC1 qm mD1 ˛im dim mD1 ˛im 'm f wj ; d0j A p0 .j / D Pn mD1 f wj ; d0j A

p0 .j / D Pn

qj d0j

qm mD1 d0m

:

(9.20)

(9.21)

(9.22)

wi are the weights or parameters of the amino acid related to the specific potential field, and f is a nonnegative potential function of w and d. For instance, qi and qj are the electronic charges, and aj and bj are the vdW field parameters for the ith aa and the jth aa, and the neighborhood relationship (truncation function ˛ij D 1) was turned on if these amino acids participate in a peptidic hydrogen bond or ˚ [199]. In this sense, the truncation of the molecular field dij < dcutoff D 5 A is usually applied to simplify all the calculations in large biological systems. The distance dij is the Euclidean distance between the C˛ atoms of the two amino acids and d0j the distance between the amino acid and the center of protein charge. Both kinds of distances were derived from the x, y, and z coordinates of the amino acids collected from the protein PDB files. All calculations were carried out with our inhouse software MARCH-INSIDE. For calculation, all water molecules and metal ions were removed. We can consider the C˛ as nodes and depict all the structure as a complex network in plain, also known as protein contact map. The protein contact map is a graph build up from a binary two-dimensional matrix; using this matrix, we are able to draw a three-dimensional graph of a protein. For two residues i and j, the ij element of the matrix is 1 if the two residues are closer than a predetermined threshold, and 0 otherwise [203]. Figure 9.2 illustrates the 3D structure of a protein ˚ In this complex network, the and the complex network for dij < dcutoff D 15:5 A. nodes are the C˛ and the edges are the bond between two adjacent C˛. The values can be calculated for different protein regions. These regions (R) called “orbits” are defined with respect to the distance from the AA ˛-carbon to the protein center of mass d(j) and the largest of these distances dmax (j). In the present study, five values were used by default for the parameter orbit D 0, 1, 2, 3, 4, considering aa with a ratio r D 100d(j)/dmax (j) ranging between the following limits: 0 D orbit0 < 25

222


Fig. 9.2 Protein 1CO4 3D structure (a) and complex network (b) used to calculate k C (j) values


223

D orbit1 < 50 D orbit2 < 75 D orbit3 < 100% or all of the AAs together, orbit4. As the orbits are related to the position of the AA with respect to the center of the protein, these are named as orbit0 D core, orbit1 D inner, orbit2 D middle, and orbit3 D outer. 2.4.3 Data Analysis, Results, and Discussion In this experiment, we used LDA to seek for two linear discriminant functions. All the independent variables are standardized before model construction. All statistical calculations were carried out with the package STATISTICA 6.0 [193]. The quality of the LDA analysis was determined by examining Wilks’ also known as U-statistic (U), Fisher ratio (F), and the p-level (p). We also inspect the ratios of good classification. The first function can classify single-point mutated proteins according to their thermal stability. The criteria used for deciding whether a protein is classified as stable or unstable were the same previously reported [88]. A dummy variable (stab) is selected to encode the studied property: stab D 1 for stable mutants and stab D 1 otherwise. Protein mutants used in this work were built as in previously published paper by Zhou and Zhou [215]. All PDB codes for proteins that were used along with the AA residue to be mutated, its number within the protein sequence, the AA introduced upon mutation, and other useful information are provided as supplementary information. All three-dimensional protein structures are extracted from the Protein Data Bank database [20]. After mutation, protein structures were energy minimized using HyperChem package [117]. As in the previous experiments, LDA [138, 201] was the preferred technique for this classification problem. Additionally, we tested the model stability and robustness by a re-substitution methodology of cases in four prediction series. For the stability models-based 3D entropy to classify protein mutants according to thermal stability, the most significant variables were the fifth-order entropy corresponding to the middle orbit, 5 C (outer), and the zero-order entropy of protein surface 0 C (surface). Introducing a new variable does not produce a significant improvement of the model. The best model found correctly classifies 293 of 300 (97.7%) stable proteins and 140 of 173 (80.9%) unstable mutated proteins for training series. For the prediction series, 72 of 75 (96.0%) stable mutated proteins are well classified. The obtained discriminant equation is the following: T S score D 1:0101 5 C .middle/ 2:147 0 C .outer/ 1:014 N D 395 U D 0:54 Chi sqr D 286:9 p level < 0:001:

(9.23)

The output of the model, TS-score, is a real value variable that scores the predicted thermal stability (TS) for the protein (see Table 9.4). On the other hand, we developed a second study on the same experiment to demonstrate the utility of entropy measures for the prediction of nonphysicochemical property such as the thermal stability (biological property). We selected the anticancer action of proteins by means of a metal-cluster

224


Table 9.4 Some examples of predicted protein thermal stability IDa

Mutantb

p(hs)c

Traind

IDa

Mutantb

p(hs)c

Traind

IDa

2CI2 1URN 1URN 1URN 1URN 1URN 1URN 1URN 1URN 1URN 1URN 1URN 1URN 1RTB 1RTB 1SAR 1RTB

K21A I12V I84V I40A I58A L17A V45A L26A I14A L69A F34A I84A L30A P93G P93A N39D P93S

0.90 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.07 0.07 0.03 0.07

In In In Out In In In Out In In In Out In In In Out In

1STN 2ABD 1STN 1STN 2CI2 1STN 2CI2 1STN 1BPI 1CYO 1CYO 1STN 1CYO 1IOB 1IOB 1CYO 1STN

T13G Y73A D95G D95A T22G Y91G D42A Y91A Y35G F35L V45G Y93A F35H K97G T9A V61E Y91S

0.91 0.95 0.91 0.91 0.79 0.91 0.79 0.91 0.97 0.72 0.72 0.90 0.72 0.93 0.93 0.72 0.92

In Out In In In In In Out In In In Out In In In In In

1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD 1LYD

N116D L99M K48A N144D Q123A V131E T115A L39A T59S A93S S44A S44G N40D Q105A A82S K60P L118M D20T A49S

0.05 0.06 0.05 0.05 0.05 0.06 0.05 0.05 0.05 0.06 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05

In In Out In In In In In Out In In In Out In In In In In Out

Mutantb

p(hs)c

Traind

1BNI S91A 2CI2 I48A 2CI2 V79A 2CI2 I49A 2CI2 L51A 1BNI N58A 2CI2 I49G 2CI2 L68A 2CI2 V66A 1VQB F68V 1BNI S92A 1VQB V35A 2CI2 F69A 2CI2 P80A 1VQB V63C 1VQB V43T 1VQB M77A

0.58 0.83 0.83 0.83 0.83 0.58 0.83 0.83 0.83 0.61 0.58 0.61 0.83 0.83 0.60 0.63 0.60

In Out In In In Out In In In In In Out In In In Out In

1LZ1 1LZ1 1LZ1 1LZ1 1C9O 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1 1LZ1

0.35 0.36 0.34 0.35 0.04 0.35 0.34 0.37 0.35 0.34 0.35 0.35 0.35 0.35 0.35 0.35 0.35 0.34 0.35

Out In In In In In Out In In In Out In In In In In Out In In

Near wild-type (nwt) mutants 2RN2 2RN2 1BPI 2RN2 1BPI 2RN2 2RN2 2RN2 2RN2 1DDR 2RN2 2RN2 1DDR 1DDR 1DDR 2RN2 2RN2 2RN2 2RN2

E48A R41C L6A D134T P8A S68A K95N K95G K95A V75C E48D D70A V75S V75A V88A D10S D134S D10A D134A

0.66 0.67 0.97 0.67 0.97 0.66 0.67 0.67 0.67 0.28 0.67 0.66 0.28 0.28 0.27 0.66 0.66 0.66 0.66

Out In In In In In Out In In In Out In In In In In Out In In

I56L A92S Y45F H78G E46A I106A T52A A9S P103G S82A V110G V110A V74A V130A I59L I106V S24A Y20F I23V

a

ID of the wild type protein in the Protein Data Bank Mutation that produce the new mutant from the respective wild type protein, mutations are given as follows: first the one letter code for the wild type protein amino acid, later the amino acid position, and finally the one letter code for the new amino acid c Probability of High Stability with which the model classify the protein mutant d The final classification in the training set b

mediated DNA-cleavage mechanism. The amino terminal Cu(II)- and Ni(II)binding (ATCUN) motif is a small metal-binding site first characterized in albumin and found in other naturally occurring proteins. This motif, formed from NH2 -aa 1-aa2-His3, participates in DNA cleavage and has been shown to have antitumor


225

activity [112]. Knowledge of metal-binding geometry helps in the design of metalbinding peptides and in understanding the mechanisms of metal-mediated functions. Since the N-terminus region of ATCUN-containing proteins is highly disordered, geometrical features cannot be derived easily from the protein structures. However, the metal-binding site is thought to involve the free N-terminal NH2 group from residue aa1, the next two peptide nitrogen atoms from residues aa2 and His3, and a nitrogen from the imidazole group of His3. Such a site has been simulated in the structure of the copper-binding peptide Gly-Gly-His-N -methyl amide, where the four nitrogen atoms form a distorted square planar arrangement [131]. Sankararamakrishnan et al. used distance constraints derived from this ATCUN peptide to search ATCUN-like motifs in 1,949 proteins. Only approximately 1.9% and 0.3% of histidines are associated with partial and full ATCUN-like geometric features, respectively. None of the ATCUN-like motifs occurs in the middle of the ˛-helix or ˇ-strand. In the previous work, a list of ATCUN-like motifs containing proteins was reported [186]. In this second study, the best model found correctly classifies 74 of 102 ATCUN-like proteins (72.5%) and 575 of 798 non-anticancer proteins (72.1%) in training series. In prediction series, the model correctly classifies 26 of 33 ATCUN-like proteins (78.8%) and 191 of 262 non-anticancer proteins (72.9%). The obtained discriminant equation is the following: AC-score D 0:804 0 C .core/ C 0:457 3 C .inner/ C 0:358 0 C .total/ 12:426 N D 900 Chi-sqr D 305:9 p-level < 0:001:

(9.24)

AC-score, the output of the model, is a real value variable that scores the predicted anticancer activity for the protein. The obtained results confirm the capability of 3D entropies to predict both physicochemical and biological properties of proteins. In particular, we demonstrated that these protein structural properties are related to the thermal stability of mutant proteins and potential enzymatic DNA-cleavagemediated anticancer action. The combined use of both models may become an interesting alternative to design stable mutants with improved biological activity.

2.5 Experiment 5 2.5.1 Introduction of Lattice Networks for DNA–Drug Docking Problem Several authors have used pseudo-folding lattice hydrophobicity-polarity (HP) models to simulate protein folding by making simulations that optimize the lattice structure and resemble the real folding [2, 19, 34, 107, 108, 124, 197, 214]. However, we can choose notably simpler pseudo-folding rules to avoid optimization procedures and notably speed up the construction of the lattice. In this sense, useful graph representations based on 2D coordinate systems of DNA, RNA, and/or protein sequences have been introduced by Gates [74], Nandy [163], Leong [141], Randic, Guo, and Basak [178]. We call these graph representations, sequence pseudo-folding

226


lattices since they look like lattice structures. In fact, we are virtually forcing a sequence to fold in a certain way which does not necessarily happen in nature. In general, these TIs characterize the distribution of amino acids or nucleotides along the sequence but also encode higher order information. Thus, lattice pseudo-folding TIs can be used in protein QSAR [83,102]. Our group, have used different MARCHINSIDE TIs of pseudo-folding lattice-like networks to predict different protein or DNA/RNA functions. For instance, we have used stochastic pseudo-folding spectral moments for the prediction of the ribonucleases [5] and Dyneins [48]. In other works, we used Markov chain pseudo-folding electrostatic potentials to predict polygalacturonases [3] or human colon and breast cancer biomarkers [204]. All these MARCH-INSIDE pseudo-folding TIs can be calculated when we sum the respective indices for each node of the graph. In this way, we can also sum local indices equivalent to the Markov entropy centrality of lattice nodes k C (j) to obtain pseudo-folding lattices entropies. All the above-mentioned values were used recently for the prediction of the mycobacterial promoters and compare entropies, spectral moments, and pseudo-folding electrostatic potentials [168]. In any case, if we understand sequence as a type of input data, we do not have to limit the application of pseudo-folding lattice network method to proteins, DNA, or RNA sequences. Elaborating this line of thinking, we have proposed pseudofolding lattice network representations of mass spectroscopy outcomes typical of blood proteome samples that contain many proteins. For instance, we have constructed lattice network representations for mass spectroscopy results obtained from blood proteome samples typical of drugs causing cardiotoxicity (see details in [43]). After the calculation of the sum of the Markov entropy centrality k C (j) for the nodes of each sample, we used them to find a new type of classifier. The model connects the k C (j) values of the blood proteome mass spectra with the probability of appearance of drug cardiotoxicity. This new type of model was called quantitative proteome–property relationships (QPPR) in analogy to QSAR or QSPR [44, 45]. We have used the same lattice-network TIs also for the prediction of the human prostate cancer [69]. The success of this strategy encouraged us to consider other classes of sequence data and solve different problems. For instance, molecular dynamics (MD) trajectories are another type of time-series data (sequential data) from simulation runs. Since the advent of MD in bioscience with the study by McCammon et al. on the dynamics of the bovine pancreatic trypsin inhibitor, MD has become the by foremost well-established, computational technique to investigate structure and function of biomolecules and their respective complexes and interactions [128, 151, 152]. The analysis of the trajectories resulting from the integration of the equations of motions in MD remains, however, the greatest challenge and requires a great deal of insight, experience, and effort. In a recent and very important work, Hamacher [109] proposed a new, theoretical sound, and versatile analysis procedure that provide scientists with a semi-quantitative invariant measure to compare various scenarios of their respective simulations. Considering that the Markov entropy centralities k C (j) of the nodes of a lattice are also invariants, we decided to explore here the use of this indices to describe MD trajectories.


227

2.5.2 Markov Entropy Centrality k C .j / for Lattices of DNA–Drug Energy Profiles The MARCH-INSIDE approach is extended to the study of lattice-like representation energy profiles obtained in DNA–drug docking studies. The key of the method we propose is the regrouping into four groups of the energy values Es (MD) obtained for different steps (s) of one molecular dynamics (MD) trajectory, after docking one drug with the DNA. These four groups characterize the deviation of the energy value Es (MD) from the average energy of the same DNA–drug complex in other steps (MD-average); or the deviation from average energy values of the same step for other drugs (step-average). First, the values of energy for the MD profile of one DNA–drug complex is placed in a Cartesian 2D space starting with the first energy value at the coordinates (0, 0). The coordinates of the successive energy values are in a similar manner than it can be used for a DNA or proteins [3]: (a) Increases in C 1 the x axis; if Es (d) > MD-average and Es (d) > Step-average (rightwards-step) or (b) Decreases in 1 the x axis; if Es (d) > MD-average and Es (d) < Step-average (leftwards-step) or (c) Increases in C 1 the y axis; if Es (d) < MD-average and Es (d) > Step-average (upwards-step) or (d) Decreases in 1 the y axis; if Es (d) < MD-average and Es (d) < Step-average (downwards-step). Second, the method uses the matrix 1 …, which is a squared matrix that characterizes the MD profile embedded into the lattice-like graph. Please note that the number of nodes (n) in the graph may be equal or even smaller than the number of steps given to obtain the MD profile. The same happens for the amino acids or DNA bases in the polymeric chain. Thus, the matrix 1 … contains the probabilities 1 pij to reach a node ni moving throughout a walk of length k D 1 from other node nj : P Ej .d / 1 E .d / s s2j D0j D0j P D P : pij D P (9.25) n n Es .d / 1 mDs ˛il D0s mDs ˛is s2j Es .d / D0s Ej (d) is the sum of all energy values of the steps Es (d) that overlap on the same node j. The parameter ˛ij is equal to 1 if the nodes ni and nj are adjacent in the graph, and equal to 0 otherwise. The value D0j gives the geometric location of the node and represents the Euclidean distance between the node and the center of coordinates. Later, it is straightforward to realize the calculation of total entropy obtained as the sum of the entropy centrality values for each node k C in analogy to all the previous experiments [3, 81] (see Fig. 9.3): k

C .L/ D

X j

k

C .j / D k

n X j D1

k

pj log k pj :

(9.26)

228


Fig. 9.3 DNA–drug complex (a) and MD profile lattice network (b) used to calculate the k C (L) values

2.5.3 Data Analysis, Results, and Discussion Using the values of k C (j), as defined previously, we calculated the total Markov entropy values for the lattices k C (L) of many DNA–drug complexes. With these values, we can obtain an LDA classifier to discriminate DNA–drug complexes of active drugs from those of nonactive drugs or non-optimal random MD profiles that act as decoys for the model. The model is a score function that may guide DNA– drug docking studies. The best model we found was the following:


229

MD score D 4:1 0 C .d / C 9:3 4 C .d / 140:1 10 C .d / C 2:9 0 C .L/ C0:2 10 C .L/ 2:7: n D 316 Rc D 0:74 U D 0:45 p < 0:01

(9.27)

The output of the model, MD-score, is a real value variable that scores the predicted goodness of fit for one MD trajectory. The model includes both entropy values k C (d) of the molecular graph of the drug (see Experiment 1) which incorporate only atom–atom connectivity information and entropy values k C (L) of the lattice derived for the DNA–drug MD profile. We selected linear discriminant analysis (LDA) [65,202] to fit the discriminant function as implemented in STATISTICA 6.0 [193]. Forward-stepwise algorithm was used for variable selection [42, 200]. The strength of the correlation was determined by the canonical regression coefficient (Rc), and the statistical significance of the LDA model was determined with U and p. All the variables included in the model were standardized to bring it into the same scale. Subsequently, a standardized linear discriminant equation that allows to compare their coefficients is obtained [139]. We also inspected the percentage of good classification, cases/variables ratios (r parameter), and number of variables to be explored to avoid over-fitting or chance correlation [200]. The model was trained with a training series and later validated with and external validation series. In training series, the model correctly classifies 75 of 79 (specificity D 94.9%) optimal and 212 of 237 (sensitivity D 89.5%) non-optimal MD trajectories. In external validation series, the model correctly classifies 27 of 27 (specificity D 100%) optimal and 71 of 79 (sensitivity D 89.9%) non-optimal MD trajectories. This results represent total accuracy D 90.8% and 92.5% in training and validation, respectively.

2.6 Experiment 6 2.6.1 Introduction to Bacterial Co-Aggregation Nearly 40 years ago, Dr. R.J. Gibbons made the first reports of the clinical relevance of what we know as bacterial biofilms when he published his observations about the role of the polysaccharide glycocalyx formation on teeth by streptococcus mutants [37]. As the clinical relevance of bacterial biofilm formation became increasingly apparent, interest in the phenomenon exploded. Studies are rapidly shedding light on the biomolecular pathways leading to this sessile mode of growth, but many fundamental questions remain. Four potential incentives behind the formation of biofilms by bacteria during infection are considered: protection from harmful conditions in the host (defense), sequestration to a nutrient-rich area (colonization), utilization of cooperative benefits (community), biofilms normally grow as biofilms, and planktonic cultures are an in vitro artifact (biofilms as the default mode of growth) [121]. In this sense, co-aggregation has been recognized as a very important

230


step in the process of biofilm formation. Specifically, co-aggregation may enhance biofilm development in the human oral cavity [136]. Co-aggregation interactions could enhance the development of biofilms in fast-flowing water systems [59] and mediate the integration of pathogens into biofilms [29]. Ultimately, co-aggregation interactions could influence the bacterial diversity of freshwater biofilms. On the other hand, co-aggregation interactions can be studied as interaction networks (co-aggregation networks). In general, biological networks such as metabolic networks [123] and protein interaction networks (PINs) [26] share important structural features with other real-world networks in dissimilar fields ranging from the Internet to social networks [6, 195]. These networks can be used to study a wide variety of biological endpoints such as the robustness of food webs against species loss [64] or identification of essential proteins within the PIN of yeast [63]. Hence, the representation of this type of interactions as a network is an interesting goal intended to further studies related to biofilms formation. Computational methods for predicting interactions have been evolved in the past years, with completely new approaches and sophisticated “mining” of existing interaction data to infer additional interactions. One new trend was to study the tendency of interacting proteins to exhibit similar phylogenetic trees [77]; quantitative algorithms for assigning interaction partners involved analyzing trees of families of interacting proteins, such as a ligand and receptor tree, and finding proteins that occupy similar positions in two trees [75, 176]. Phylogenetics played an important role in evolutionary biology [113]. Today it is not uncommon to see phylogenies applied in fields far removed from evolutionary biology [9, 146]. In microbiology, they have found a practical use in tracing routes of infectious disease transmission and in identifying the relationship of pathogens, such as the New Mexico Hantavirus [115], studying an evolving virus such as HIV [39], predicting co-regulated genes in the bacterium Bacillus subtilis [122], finding tree-based protein classification [140] as well as phylogenetic approaches in linguistic and cultural evolution to increase our understanding of human prehistory and adaptation [145]. Considering the potentialities of phylogenetics is interesting to make use of phylogenetics trees for the reconstruction of bacterial co-aggregation network to shed light insight of the bacterial co-aggregation phenomenon. However, recent studies of co-aggregating freshwater biofilm bacteria have been demonstrated that the co-aggregation often occurs between bacteria that are taxonomically distant (intergeneric co-aggregation) and occasionally between strains belonging to the same species (intraspecies co-aggregation) [180]. As a consequence, it becomes very difficult to predict the occurrence of coaggregation by direct visual inspection of the phylogenetic tree. In general, many researchers use the qualitative comparison of tree topology to gather different rules in phylogenetic analysis [110, 175]. This evidence certainly necessitates the development of tree topological indices (TTIs) that are able to compare a trees’ topology both quantitatively and qualitatively as well as connect it to other relevant properties. With this purpose, we use here the representation of the phylogenetic tree as a graph. It means that we represent both the bacterial species and the evolutionary stages as nodes connected by evolutionary steps represented by edges.


231

In principle, different kind of graph or networks and their indices has been used to describe very different type of chemical and biological systems [25]. Based on this kind of supposition, we can use local measures of the connectedness of the graphs such as TIs or node centralities to describe the bacterial co-aggregation. In recent years, many TIs derived from graph representations have been introduced to study structural graphs at the molecular and macromolecular level [32, 33, 142]. For a more detailed discussion, the reader may see recent in-depth reviews in the field [66, 72, 83, 102]. We propose here a new kind of Ct (j) and/or TIs which may be comprehended as entropies of divergence (k C (a,b)) of the evolution pathway that resulted in the formation of the species a and b. These new centralities are TTIs that can be calculated using a Markov Model (MM) at different steps of bacterial coevolution within a 16S rRNA genes phylogenetic tree. In any case, the new TTIs may be applied in principle to quantitatively describe the topology of any kind of trees previously published for phylogenetic analysis [198, 206]. The selection of entropy measures was justified considering the several successful applications reported for this kind of parameters for the study of diverse complex systems; see for instance previous works of Graham [104,105]. Next, we used the k C .a; b/ values to develop for the first time a quantitative sequence-interaction relationship (QPHPR) model that can predict the probability of two bacteria species to co-aggregate, given its 16S rRNA sequences for a large set of 138 pairs of non-co-aggregating bacteria and 111 pairs of co-aggregating ones. The corresponding bacterial co-aggregation network was reconstructed using the above-mentioned methodology. Finally, the real and the reconstructed co-aggregation networks were analyzed to distinguish their scale-free or random architecture based on their respective distribution (bell curve or power law) [15]. 2.6.2 Markov Entropy Centrality k C .a; b/ for Bacterial Phylogenic Tree We used a database that was previously experimentally studied and reported in the literature to derive the present modeling of bacteria co-aggregation [181]. In Fig. 9.4, we illustrated the phylogenetic tree used in this previous experimental work. In the tree, the names of the investigated bacteria species appeared in boldface style and other bacteria species not investigated by these authors appear in normal style. We focused our attention herein on the evolution divergence pathway followed by every pair of two bacteria species a and b after k evolution steps moving forward (and also backward) from the root of the tree (ancient common bacteria species) to the final or current species. Our hypothesis is that entropy measures of the different steps given by both species in this pathway express certain degree of divergence of two bacteria and may be then connected to their co-aggregation. The algorithm based on the MARCH-INSIDE approach is used here for the first time to codify phylogenetic information, but has been applied previously to the study of proteins [3, 11, 86, 88, 91, 95, 96, 98, 100, 185]. Here, the classic Markov matrix MARCHINSIDE approach [11] has been adapted to characterize phylogenetic information in the following way.

232


Fig. 9.4 Evolution phylogenetic tree of bacteria species used to calculate the 2 C (a,b) values

First, we need to construct the matrix 1 … [see (9.1)]. This matrix is built up as a square matrix (n n). The matrix 1 … contains the evolution transition probabilities 1 pij to reach a node ni , moving from a node nj inside the phylogenetic tree throughout a walk of length k D 1 (one evolution step). Let ıj be the number of


233

possible evolution steps that the species may give from node nj to ni (degree of the phylogenetic tree node) and given that ˛ij D 1 if and only if the two nodes ni and nj are neighbors placed at topological distance k D metricconverterProductID1 in1 in the phylogenetic graphs (it means separated by one evolution step) and ˛ij D 0 otherwise, we can calculate: 1

˛ij ıj pij D Pn : lD1 ˛il ıl

(9.28)

In addition, we use the absolute initial probabilities vector 0 [see (9.1)]. This vector contains the absolute initial probabilities k pj to reach a node ni from a randomly selected node nj . Here, we consider the initial probability inverse to the large (N, number of nodes) of the phylogenetic tree. Next, we used the theory of Markov chains to calculate the evolution entropy centrality, k C (a,b) : 0

k

pj D

C .a; b/ D

1 N X

(9.29) k

C .j / D

X

k

pj log k pj :

(9.30)

k

pj are the absolute probabilities to reach the nodes na or nb moving throughout a walk of length k from node n0 that represent the root of the phylogenetic tree. The sum runs only over the nodes that lie within the divergent evolution paths that connect the root of the tree with the nodes na and nb . In this equation, kB is the Boltzmann’s constant, which is used as a physical unit scaling value. The Chapman– Kolmogorov equations were used to calculate the vector k containing the k pj values using the vector 0 of initial probabilities (0 pj ) and the matrix 1 … with the first-step transition probabilities (1 pij ). The values of k C (a,b) were calculated with the software MARCH-INSIDE 3.0 [90] and the following equation: k D 0 k … D 0

1 k … :

(9.31)

2.6.3 Data Analysis, Results, and Discussion Using the values of 2 C (a,b) as defined previously, for all pairs of bacterial species we can attempt discriminate co-aggregating from non-co-aggregating species by fitting a simple linear classifier with the formula: BBC score D b0 C a0 0 C .a; b/ C a1 1 C .a; b/ C ::: C ak k C .a; b/ X D b0 C ak k C .a; b/: (9.32) The output of the model, BBC-score, is a real value variable that scores the possibility of existence of a bacteria–bacteria co-aggregation (BBC) relationship

234


between two species. We selected LDA [65, 202] to fit the discriminant function. In the previous equation, b0 and ak represent the coefficients of the classification function, determined by the least square method as implemented in the LDA module of the STATISTICA 6.0 software package [193]. The forward-stepwise algorithm was used for variable selection [42, 138, 200]. The statistical significance of the LDA model was determined by Fisher’s test by examining Fisher ratio (F) and the respective p-level (p). All the variables included in the model were standardized to bring it into the same scale. Subsequently, a standardized linear discriminant equation that allows to compare their coefficients is obtained [139]. We also inspected the percentage of good classification, cases/variables ratios (r parameter), and number of variables to be explored to avoid over-fitting or chance correlation [138, 200]. The model was trained using and later validated with external validation series. In order to reconstruct the bacterial co-aggregation interactions network, we randomly selected 104 co-aggregating and 83 non-co-aggregating pairs of bacteria to train the first model. A supplementary material available upon author’s request contains the 16S-rRNA phylogenetic tree used to derive the kth Markov evolution divergence entropies. It also contains the predicted classification and posterior probability of co-aggregation according to the QPHPR model. The values of 2 C (a,b), residuals and leverages for co-aggregating/non-co-aggregating pairs of bacteria are used for training and validation. This model is able to correctly predict the 88.24% of the co-aggregation interactions between the 187 pairs of bacteria used with only one parameter. It also correctly predicts all the non-co-aggregating interactions and the 73.49% of the co-aggregating interactions (see Table 9.5). In addition, the receiver operating characteristic curve (ROC curve) obtained with an area under the curve of 0.89 indicates that the model is not a random but a statistically significant classifier. An ROC curve [216] plots the sensitivity versus one minus the specificity. An ideal classifier hugs the left side and top side of the graph, and the area under the curve is 1.0. A random classifier should achieve approximately 0.5. The resulted equation of the model is: BBC score D 2:326 2 C .a; b/ 0:398 F D 142; 01

U D 0:566 p < 0:00005

(9.33)

The next step is to find out whether the basic assumptions of LDA are fulfilled [194]. As the names imply, LDA establishes a linear, additive relationship between the predictor TTIs [2 C (a,b)] and the response variable. However, other functional relationships between TTIs and co-aggregation (quadratic, cubic, logarithmic, and inverse) were studied to select the relationship that best describe the phenomenon. Correct selection of the functional form was checked by visual inspection of the distribution of the standardized residuals for all co-aggregation interactions pairs. The homocedasticity was checked in a similar manner to the functional form by plotting the squares of the standardized residuals versus the values of 2 C (a,b) for each


235

Table 9.5 Examples of results for some co-aggregating bacteria pairs used to train the model (Species a / species b) Aeromonas sp./A. hydrophila B. sorensis / L. aquatica C. aquaticum / A. delafieldii C. aquaticum / L. aquatica C. aquaticum / L. aquatica F. columnare / B. licheniformis F. columnare / L. aquatica F. columnare / Shewanella sp. L. aquatica / B. licheniformis L. aquatica / kocuria rhizophila L. aquatica / Methylobacterium sp. L. aquatica / Vivrio sp. L. aquatica / Methylobacterium sp. A. delafieldii / B. licheniformis L. aquatica / L. aquatica L. aquatica / K. rhizophila S. maltophilia / B. licheniformis S. maltophilia / L. aquatica S. maltophilia / Pseudomonas sp. S. maltophilia / S. intermedius a

P(C)a 0.9 0.8 0.5 0.9 0.9 0.8 0.5 0.5 0.7 0.9 0.9 0.7 1.0 0.8 0.9 0.9 0.7 0.7 0.9 0.9

2

C (a; b) 1168.0 1246.0 1401.0 1090.0 1090.0 1246.0 1401.0 1401.0 1324.0 1168.0 1090.0 1324.0 1012.0 1246.0 1168.0 1168.0 1324.0 1324.0 1090.0 1168.0

Residual 0.2 0.3 0.5 0.1 0.1 0.3 0.5 0.5 0.4 0.2 0.1 0.4 0.1 0.3 0.2 0.2 0.4 0.4 0.1 0.2

Leverage 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.02 0.01 0.01 0.01 0.01 0.01 0.01 0.01

P(C) is the predicted probability of co-aggregation

co-aggregation interaction pair. In both cases (correct specification of the functional form and homocedasticity), if a systematic pattern is observed, the respective hypotheses must be rejected. Note the systematic (parabolic in most cases) pattern observed in all the different models studied; hence the homocedasticity hypothesis is rejected in all the cases. However, concerning to the functional form, despite no systematic pattern is observed in any alternative model, the linear one shows the most disperse distribution corresponding to a better fitting to the chosen (linear) functional form. The hypothesis of normal distribution of the residuals is rejected for all the alternative models tested according to the results of the Kolmogorov– Smirnov and Lilliefor tests for normality (D and p statistic values) [194]. However, by a visual inspection of the frequency histograms of residuals, it is possible to note that a linear model is best fitted to the normal distribution evidencing their slight deviation from the normal distribution. Finally, the mean value of the residuals must be zero which means that the exclusion of the residual (error) term on the equation is justified. In this sense, the lowest value is shown by the linear model. In summary, there is no model that fully fulfills the parametrical assumptions. However, the linear one is the model that best fits to a chosen functional form and shows the lowest mean residual value and the lowest deviations from the normal distribution. Only the homocedasticity assumption is clearly violated. Bearing this in mind, we decided to test the predictive ability of the linear model. The predictability and, consequently, the ability of the

236


model to reconstruct the bacterial co-aggregation network were tested using the remaining 34 and 28 pairs of non-co-aggregating and co-aggregating bacteria never used to fit the model, respectively. The model correctly predicts the 87.09% of the co-aggregation interactions in the test set. It also correctly predicts all the non-coaggregating interactions and the 71.09% of the co-aggregating interactions. The area under the curve for test set is clearly over 0.5 (0.88). On the other hand, considering the difficulties of the different models performed by parametric methods to fully fulfill their assumptions is worthy to try other nonparametric techniques in solving the problem. To accomplish this goal, we use 45 machine learning algorithms for classification implemented in WEKA (Waikato Environment for Knowledge Analysis) software [207,211]. No significant improvement was achieved with any one of the classification schemes applied to the data. Percentages of correctly predicted co-aggregation interactions equal or lower than 57.75% were reached by 25 of 40 classification schemes applied. No significant improvement (%T D 88.77; %Co-agg. D 74.70; %NonCo-agg. D 100) on the performance of the LDA model was achieved by 10 of 45 classification schemes in train set. The same performance was achieved on the test set. On the other hand, 10 of 45 schemes show identical performance to the LDA model in training and test sets. Details about the application of the 45 machine learning algorithms classification schemes found can be included in the supplementary material related to this work. Finally, in this experiment we report the QPHPR prediction of the bacterial co-aggregation complex network. The real bacterial co-aggregation interactions network was represented using the Centralities in Biological Networks (CentiBin) software [137], an application for the calculation and visualization of centralities for biological networks. The adjacency matrix related to the observed co-aggregation interactions (if the pair of bacteria co-aggregate then the matrix element is 1, otherwise 0) was used as input. For the reconstruction of the corresponding network, we use the adjacency matrix generated by the classifications of the LDA model as input for the CentiBin software. The real and the reconstructed bacterial coaggregation interaction networks are shown in Fig. 9.5. Additionally, the method proposed here is efficient on the reconstruction of the network if reasonable to expect that both real and reconstructed share the same architecture. Thus, we decide to determine whether our networks show a freescale (normal distribution) or power-law (exponential distribution) architecture [15]. According to the hypothesis tests (Kolmogorov–Smirnov and Lilliefor), the real and reconstructed networks do not present normal or exponential distribution, but show similar distributions. Based on the respective D values, both networks are closer to the free-scale architecture. In conclusion, the TTIs called Markov evolution divergence entropies k C (a,b) were introduced as phylogenetic bio-descriptors for bacteria–bacteria interspecies similarity. Specifically, the second-order entropy k C (a,b) was sufficient to develop a QPHPR model able to correctly predict the 88.24% of the co-aggregation interactions between the pairs of bacteria used for training, exhibiting a predictive ability of 87.09%. Numerous non-parametrical methods were evaluated to overcome


237

Fig. 9.5 Observed (a) bacterial co-aggregation network vs. network predicted (b) with model based on centrality values 2 C (a,b) of the phylogenetic tree

238


the performance of the LDA-based QPHPR model but not one justified to discard the simple linear form. The present results introduces the first quantitative rule to reconstruct bacterial co-aggregation interactions networks based on RNA sequences, which constitutes a step of relevance in the bioinformatics approach to bacteria biofilms formation. We can understand the use of numerical indices derived from phylogenetic tree as an alternative to models based on parameters derived directly from protein or nucleic acid sequences. Using the sequence directly is faster and avoids constructing the phylogenetic tree. See for instance important works on the application of TIs of protein sequences and RNA secondary structure [147, 148]. In any case, the phylogenetic tree could hide interesting biological information that can be unraveled with TTIs, as in this case. The use or not of phylogenetic tree TTIs for QPDPR modeling is still under debate, and opening this line of thinking is one of the merits of the present work.

2.7 Experiment 7 2.7.1 Introduction to Host–Parasite Interactions With the availability of complete genomic sequences of various hosts and pathogens, together with breakthroughs in proteomics, metabolomics, and other experimental areas, the investigation of host–pathogen systems at multitude levels of details has come within reach. Unlike traditional biological research that focuses on a small set of components, systems biology studies the complex interactions between a large number of genes, proteins, and other elements of biological networks and systems; host–pathogen systems biology examines the interactions between the components of two distinct organisms, either a microbial or a viral pathogen and its animal host or two different microbial species in a community [70]. In these networks, the nodes are the microbial or host species, and the edges (arcs) indicate the presence of host-pathogen relationships (HPRs). To construct these networks, it is necessary to experimentally determine (or discard) the existence of specific relationship between two species. This may result in a time-consuming task with high material cost. In this sense, the development of new computational models to predict HPRs becomes of the major interest. We can infer these HPRs from the sequence of both the parasite and the host. The construction of a phylogenetic tree that expresses the co-evolution of both parasites and host may be of great help. Unfortunately, many times direct inspection of the Tanglegram does not give a clear discrimination between interacting and noninteracting species. Anyhow, we can mention that the information to discriminate between HPRs from noninteracting species is not evident but hidden some way in the complicated evolution path that has given rise to the existing species. In this sense, we can expect that some indices may contain information about the branching of the tree, and we can try to use them in the recognition of HPRs. For it, we can use entropy TTIs (see previous experiment) and/or entropy node centralities k C (j) of the phylogenetic tree, which are TIs of local-evolution steps. However, until now these indices have not been


239

used to predict parasite–host interactions. In the following two sections, we shall discuss the calculation of TIs and Ct (j) for RNA secondary structure and the use of these indices to predict host–parasite interactions. 2.7.2 Markov Entropy Centralities k C .j / for Phylogenetic Host–Parasite Networks This type of network have three different subnetworks: subnetwork (1) is a phylogenetic tree of the evolution of parasite species, subnetwork (2) is a phylogenetic tree of the evolution of host species; both subnetworks together form a co-evolution or Tanglegram tree. In the subnetworks (2) and (3), the edges connect ancestors with new hypothetical or final (real) species. Conversely, in the subnetwork (3), the edges express only the existence of HPRs between one host and one parasite species. This subnetwork (3) contains only the nodes of subnetworks (1) and (2) corresponding to present species but omit node representing ancestors’ evolution. We used a database previously experimentally studied and reported in the literature to derive the present modeling of for HPRs (see Fig. 9.6). Conversely to the bacteria co-aggregation study (see the previous section), we focused our attention on the entropy of co-evolution convergence pathway followed by every pair of HPRs. Our hypothesis is that entropy measures of the different steps given by both species in this pathway express certain degree of co-evolution convergence and may be connected then to their interaction. The algorithm based on the MARCH-INSIDE approach [3, 11, 86, 87, 89, 90, 98–102, 185] is used for the first time to codify the phylogenetic information. As in the previous section, the classic Markov matrix MARCH-INSIDE approach [11] has been adapted to characterize phylogenetic information in a similar way with the Sect. 2.6.2 (9.28)–(9.31). Next, we used the theory of Markov chains to calculate the evolution path entropy centrality (k C (q)) for the host q D H.

2.7.3 Data Analysis, Results, and Discussion We can attempt to discriminate HPRs from noninteracting species using the values of k C (j) previously defined as inputs. As in other experiments, we selected LDA [65] to fit a discriminant function. The equation of the best model found was the following: HPR score D 1:10 1 C .p/ 8:23 Œ5 C .f / C 5 C .p/ C8:17 5 C .f / 5 C .p/ 0:84: N D 260

U D 0:6409 F D 47:813

p < 0:001

(9.34)

The output of the model, HPR-score, is a real value variable that scores the HPR interaction between one parasite and one host species. In this equation, the symbols

240


Fig. 9.6 Host–parasite co-evolution phylogenetic tree (Tanglegram) used to calculate the k C (j) values

used for different centralities are: if j D p, we obtain k C (p) D Markov entropy centrality of order k for the evolution path of parasite; and if j D f, k C (f) D Markov entropy centrality of order k for the evolution path of the host fish species. The descriptors are calculated for as the sum of all centrality values of the same order for all nodes contained in the path going from the root of the phylogenetic tree to the node representing the actual species. In total, 48 of 60 HPRs (sensitivity D 80.0%) and 175 of 200 noninteracting species pairs (specificity D 87.5%) of

9 Markov Entropy Centrality: Chemical, Biological, Crime, and Legislative Networks Table 9.6 Prediction of 60 host–parasite interactions Host–parasite pairs p Host–parasite pairs P. phoxinus–borealis 0.61 L. cephalus–vranoviensis A. aspius–ramulosus 0.51 C. nasus–chondrostomi A. aspius–tuba 0.59 C. nasus–ergensi L. idus–crucifer 0.56 C. nasus–vistulae L. idus–ramulosus 0.51 R. rutilus–caballeroi L. idus–tuba 0.59 R. rutilus–crucifer L. idus–vistulae 0.44 R. rutilus–fallax A. ballerus–auriculatus 0.40 R. rutilus–nanus A. ballerus–chranilowi 0.40 R. rutilus–rarissimus A. sapa–propinquus 0.57 R. rutilus–rutili A. brama–falcatus 0.57 R. rutilus–similis A. brama–wunderi 0.84 R. rutilus–sphyrna A. brama–zandti 0.72 P. parva–squameus A. bjoerkna–cornoides 0.91 G. albipinatus–finitimus A. bjoerkna–cornu 0.91 G. gobio–cryptomeres A. bjoerkna–distinguendus 0.84 C. idellla–lamellatus A. bjoerkna–sphyrna 0.25 B. barbus–carpathicus A. alburnus–alatus 0.40 B. barbus–dyki A. alburnus–fraternus 0.99 B. barbus–malleus A. alburnus–minor 0.98 C. auratus–anchoratus A. alburnus–parvus 0.99 C. auratus–dulkeiti S. erythrophthalmus–crucifer 0.72 C. auratus–formosus S. erythrophthalmus–difformis 0.99 C. auratus–inexpectatus S. erythrophthalmus–difformoides 0.99 C. auratus–intermedius S. erythrophthalmus–izjumovae 0.99 C. auratus–vastator L. cephalus–fallax 0.72 C. carpio–achmerowi L. cephalus–folkmanovae 0.95 C. carpio–anchoratus L. cephalus–nanoides 0.95 C. carpio–extensus L. cephalus–prostae 0.98 G. cernuus–amphibothrium L. cephalus–vistulae 0.14 G. cernuus–hemiamphibothrium

241

p 0.84 0.78 0.78 0.07 0.78 0.78 0.78 0.91 0.91 0.91 0.35 0.17 0.86 0.64 0.64 0.99 0.10 0.10 0.10 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 1.00 1.00

the real parasite–host network were recognized by the model in training series (see Table 9.6). In addition, 47 of 60 HPRs (sensitivity D 78.3%) and 175 of 200 noninteracting pairs of species (specificity D 87.5%) of the real parasite–host network were recognized by the model in LOO cross-validation experiments.

2.8 Experiment 8 2.8.1 Introduction to Criminal Causality Networks One of the reasons people have difficulty in dealing with complex systems is that the linear causal chain way of thinking – A causes B causes C causes D . . . etc – breaks down in the presence of feedback and multiple interactions between causal

242


and influence pathways. One could say that complex systems are characterized by networked rather than linear causal relationships. Nevertheless, it is important to be able to reason about complex systems, make inferences about factors that contribute to current and alternative states of complex systems, and explore their possible future trajectories, especially if we wish to influence them toward more favorable futures, and away from more dangerous possibilities. Large-scale examples include ecosystems, economic systems, coupled biophysical-socioeconomic systems, integrated supply chains/industrial systems and social systems, but these remarks also apply for example to attempts to understand a physical organism as a complex system. Crime causality is a very important phenomenon in this sense. Different measures of crime causality have been developed before [55]. In this work, we introduced the Markov entropy centrality k C (j) for a node in a crime causality network. At the same time, we propose new measures of crime causality calculated as of the sum of all the k C (j) values of the same order k for all nodes placed in the shortest path (shp), connecting the original node ni (possible cause) with the final node nii (consequence). 2.8.2 Markov Entropy Centralities k C .j / for Actions in Crime Networks First, we need to construct the crime causality Markov matrix 1 …. This matrix is built up as a square matrix (n n), where n are all the actions related to the crime including the original actions (causes), the co-actions (secondary causes), and the consequence (crime). The matrix 1 … contains the transition probabilities (1 pij ) that have the action i to be the cause or at least to be occurred immediately after it in the crime than other action j. The probabilities 1 pij may be calculated using the (9.28) and (9.29). ıj represents the number of actions that occurred immediately after the action ith. In addition, we use the absolute initial probabilities vector 0 ; see (9.26). This vector lists the absolute initial probabilities k pj to reach a node ni from a randomly selected node nj . Here, we consider the initial probability inverse to the dimension (N, number of nodes) of the shp connecting ni with nii . Next, we used the theory of Markov chains to calculate the criminal causation entropy centrality k C (i,ii): X X k k k (9.35) C .i; ii/ D C .j / D pj log k pj : j 2shp

j 2shp

k pj are the absolute probabilities to reach the nodes moving throughout a walk of length k from node ni . The sum runs only over the nodes that lie within shp connecting ni with nii . The Chapman–Kolmogorov equations were used to calculate the vector k containing the k pj values using the vector 0 of initial probabilities (0pj) and the matrix 1 … with the first-step transition probabilities (1 pij ).

k D 0 k … D 0

1 k … :

(9.36)


243

2.8.3 Data Analysis, Results, and Discussion Using the values of k C (i,ii), as defined previously, for all pairs of cause(i)consequence(ii) or the same causality paths, we can attempt to discriminate determinant causes from less important causes in a crime network. We selected LDA [65] to fit the discriminant function. CC score D b0 C a0 0 C .i; ii/ C a1 1 C .i; ii/ C : : : C ak k C .i; ii/ X D b0 C ak k C .i; ii/: (9.37) b0 and ak represent the coefficients of the classification function, determined by the least square method implemented in the LDA module of the STATISTICA 6.0 [193]. Forward-stepwise algorithm was used for variable selection [42,200]. The statistical significance of the LDA model was determined by Fisher’s test by examining F and p. All the variables included in the model were standardized to bring it into the same scale. Subsequently, a standardized linear discriminant equation that allows to compare their coefficients is obtained [139]. We also inspected the percentage of good classification, cases/variables ratios (r parameter), and number of variables to be explored to avoid over-fitting or chance correlation [200]. The model was trained using and later validated with external validation series. The best model found was: CC-score D 35:36 0 C .i; ii/ C 112:80 5 C .i; ii/ 25:59: n D 47 Rc D 0:85 Chi-sqr D 72:8 p < 0:005

(9.38)

The output of the model, CC-score, is a real value variable that scores the possibility of a crime cause (CC) to be the main cause of a given crime. This model is able to correctly predict the 94.74% of the main crime causes (CC) of the 47 potential crime causes in 17 crime cases. We also obtained two additional classification functions to discriminate secondary cases of to lower degrees not reported here for reasons of space. The present model also correctly predicts 94.74% of main crime causes in Leave-One-Out (LOO) cross-validation experiments. Figure 9.7 depicts the space for the criminal causation model.

2.9 Experiment 9 2.9.1 Introduction to Legislative Networks Social network Analysis (SNA) may be defined as the disciplined inquiry into patterning of relations among social actors, as well as the patterning of relationships among actors at different levels of analysis (such as persons and groups) [28]. It provides a common approach for all those disciplines involved in social structure

244


Fig. 9.7 Canonical space for the criminal causation model

study [38, 208, 209] susceptible of network depiction. Social structure concept is merely used in sociology and social theory. Although there is not agreement between theorists, it can refer to a specific type of relation between entities or groups also can evolve enduring patterns of behavior and relationship within a society, or social institutions and norms becoming embedded into social systems. For a most complete review of SNA, see the in-depth review of Newman M entitled: The Structure and Function of Complex Networks [164]. In any case, if we take into consideration that a network is a set of items, usually called nodes, with connections between them, so-called edges [164], thus it means the representation of social relationships in terms of nodes and ties, where nodes can be the individual actors within the networks, and ties the relationships between these actors [27]. In fact, SNA is nothing new in social sciences studies; in early 1930s, sociologists already have made a social network to study friendships between school children [157]. Since the importance of network approach to social sciences highly increased, its application goes from interrelation between family members [166] to companies business interaction [155] or patterns of sexual contacts [135, 143]. Although the network approach is so pervasive in the social sciences, their application in the Law scope is still weak. Legislative–political SNA analysis deals with legislative and/or political SNs. It means SNs that describes legislative production and/or political actions. In this context, networks tools and methodologies might be useful to illustrate the interrelation between the different law types. Johnson and Orbach


245

[125] have published an interesting paper about moderate-sized legislative political network involving legislators, agency heads, lobbyists, industry representatives, and agency and legislative staff. We can use legislative–political SNA also to check the importance of a specific instrument so as the normative hierarchy respect by legislators or precedents in court decisions. For it, SNs have been also studied with classic node centrality values. For instance, Fowler and Jeon [71] reported the construction of the complete network of 30288 majority opinions written by the US Supreme Court and the cases they cite from 1754 to 2002 in the United States Reports. Data from this network demonstrate quantitatively the evolution of the norm of stare decisis in the nineteenth century and a significant deviation from this norm by the activist Warren Court. They further described a method for creating authority scores (node centralities) using the network data to identify the most important court precedents. This method yields rankings that conform closely to evaluations by legal experts, and even predicts which cases they will identify as important in the future. Taking into consideration the importance of the codification of information in this context in our opinion is very important to introduce new type of entropy centralities k C (j) for legislative–political SNA. With this aim, we used the software S2Snet [160] to construct, represent, and calculate the entropy of legislative production. 2.9.2 Markov Entropy Centralities k C.j / for Legislative Networks In this experiment, we discuss the use of star networks to represent legislative time production of financial laws. We represented the time series for the use of different laws as a one-letter code sequence. In this sequence, we use one letter to represent each specific type of rule, law, or norm introducing along the time by the legislative system of a given country. Specifically, we transform all the norm types as follows: L-Law, I-Instrument of Ratification, D-Decree-law, O-Order, P-Protocol, R-Royal Decree, C-Circular, E-European Community Council Directive, S-Resolution, G-Organic Law, A-Agreement, T-Instruction, V-Convention, and M-Regulation. From there, we can use the software S2SNet [161] to construct one Star Network that connects several branches to a central node. Only the same type of laws composes each branch. Nodes in the same branch or another are connected to each other if the respective two laws were put into practice in time one immediately after other. Once we construct the Legislative network with S2SNet, we need to calculate the legislative production entropy centralities. For this we have to construct the legislative production Markov matrix 1 …. This matrix is built up as a square matrix (n n), where n is the node representing laws of type L introduced at a given year (t). The matrix 1 … contains the transition probabilities (1 pij ) that have the legislative system to pass from introduction of law ni to law nj . The probabilities 1 pij may be calculated using the same (9.28). ıj is equal to the number of law nodes that are connected to nj in the network (law degree). In addition, we use the absolute initial probabilities vector 0 (9.40). This vector lists the absolute initial probabilities k pj

246


to reach a node ni from a randomly selected node nj . Here, we consider the initial probability inverse to the graph dimension (N, number of nodes). Next, we used the theory of Markov chains to calculate the legislative entropy centrality for given node k C (L,t), representing the law L introduced at time t. We can sum all the k C (L,t) for all type of laws at a given time t or along the time, but for a give type of law to construct different cumulative or law type group centralities such as: k

C .L/ D

X

k

C .L; t/ D

j 2L k

C .t/ D

X

X

k

pj log k pj ;

(9.39)

j 2L k

C .L; t/ D

j

X k

pj log k pj :

(9.40)

j 2shp

k

pj are the absolute probabilities to reach the law nodes moving throughout a walk of length k from every node ni . The Chapman–Kolmogorov equations were used to calculate the vector k containing the k pj values using the vector 0 of initial probabilities (0 pj ) and the matrix 1 … with the first-step transition probabilities (1 pij ).

2.9.3 Data Analysis, Results, and Discussion This method describes numerically the recurrence to different norm or group of norms or laws (legislative production of a country). We can use the entropy centralities of legislative production networks of this type to describe the past dynamics and predict the future behavior of tax law use in one country. In any case, many potential implications are still to be discovered in future research beyond this introductory work. In Table 9.7, we illustrate the behavior of different time Table 9.7 k C (t) values for Spanish Financial law system over years, see also Fig. 9.8 0 1 2 3 4 5 Law type-Yeara tb C (t) C (t) C (t) C (t) C (t) C (t) D-27/02/1946 0 1.24 1.34 1.24 1.34 1.24 1.34 L-23/07/1960 14 2.40 2.80 2.55 2.80 2.60 2.79 L-21/07/1965 19 2.89 3.18 3.02 3.17 3.07 3.17 I-27/10/1972 27 3.28 3.69 3.42 3.68 3.46 3.67 I-31/12/1975 30 3.65 3.95 3.81 3.93 3.84 3.93 R-30/12/1980 35 4.32 5.19 4.67 5.17 4.78 5.17 L-28/12/1985 40 4.77 5.72 5.07 5.72 5.19 5.72 R-16/11/1990 45 5.34 6.19 5.68 6.16 5.79 6.16 R-30/12/1995 50 6.04 6.50 6.24 6.51 6.30 6.52 R-30/12/2000 55 6.50 6.80 6.64 6.82 6.68 6.83 R-26/06/2004 58 6.79 7.03 6.90 7.04 6.94 7.05 Mean SD a b

– –

4.29 1.78

4.76 1.88

Year is the date of the period analyzed t is the time duration in years of the period

4.48 1.83

4.76 1.88

4.54 1.84

4.76 1.89


247

Fig. 9.8 S2SNet view of Spanish Tributary law system: 1946–1978 (a) and 1946–2004 (b)

cumulative entropy centrality k C (t) values for the star networks of the legislative production on the Spanish tributary system. The networks constructed reflect the cumulative changes in the recurrence to different laws in these periods. In Fig. 9.8, we depicts the S2SNet view of the cumulative time behavior of the Spanish Tributary law system over two periods: 1946–1978 (a) and 1946–2004 (b). We can observe the increase in the entropy centralities in time and the almost constant means and standard deviations.

248


3 General Conclusions This chapter is proposing the quantification of the information in terms of the entropy centrality of the states (nodes) of a Markov Chain associated with the system, represented by a network graph. A standard procedure is applied for nine cases as following: the prediction of the chemical skin sensitization with molecular graphs, the introduction of a QSAR model to predict the action of small molecules over multiple targets and to construct drug–target networks based on molecular graphs, the study of the chemical reaction, the prediction of the protein thermal stability and protein anticancer action by DNA-cleavage mechanism, the calculation of scoring functions for DNA–drug docking with lattice network representations of the molecular dynamic trajectories, the prediction of the bacterial co-aggregation and reconstruction of the bacteria–bacteria co-aggregation networks with the phylogenetic tree graphs, the evaluation of the host–parasite interaction networks using the phylogenetic co-evolution graphs, the study of the cause and consequence links in Criminal networks, and the analyses of the legislative production networks of the Spanish tributary system. Thus, we demonstrate the simplicity and applicability of the entropy of a Markov Chain associated with a network or graphs as a universal quantity in pattern recognition regardless the chemical, biological, social, or other types of systems. Our group will introduce this class of centralities/TIs in the study of genotype– phenotype relationships by creating networks of SNP linked with specific diseases such as colon cancer. In addition, the models will be obtained using the powerful techniques of multivariate statistical analysis such as variable selection, model creation by principal component regression, and partial least-squares regression. Thus, we hope to obtain models capable to help in the future diagnostic of complex diseases where the traditional methods fail. Acknowledgments C.R. Munteanu and H. González-D´ıaz acknowledge financial support of Program Isidro Parga Pondal of the funded by Dirección Xeral de Investigación e Desenvolvemento, Xunta de Galicia. S. Arrasate acknowledges sponsorships for a tenure-track research position at the University of Santiago de Compostela from the “Ikertzaileak Hobetzeko eta Mugitzeko/Perfeccionamiento y Movilidad del Personal Investigador” Program of the “Hezkuntza, Unibertsitate eta Ikerketa Saila/Departamento de Educación, Universidades e Investigación, Eusko Jaurlaritza/Gobierno Vasco.” Financial support from Gobierno Vasco (GIC07/92-IT-227-07) is also gratefully acknowledged. A. Duardo-Sánchez gratefully acknowledges Prof. Begoña Villaverde, Ph.D. and Prof. A. López-D´ıaz for financial support (project 2006/PX 207) of Department of Especial Public Law, Financial and Tributary Law Area, Faculty of Law from University of Santiago de Compostela (Spain), which was funded by Xunta de Galicia. This work is supported by the “Ibero-American Network of the Nano-Bio-Info-Cogno Convergent Technologies,” IberoNBIC Network (209RT0366) funded by CYTED (Ciencia y Tecnologa para el Desarrollo) and by the COMBIOMED Network, the grant (Ref. PIO52048 and RD07/0067/0005), funded by the Carlos III Health Institute.


249

References 1. Abhiman, S., Sonnhammer, E.L.: Large-scale prediction of function shift in protein families with a focus on enzymatic function. Proteins 60(4), 758–768 (2005) 2. Agarwala, R., Batzoglou, S., Dancik, V., Decatur, S.E., Hannenhalli, S., Farach, M., Muthukrishnan, S., Skiena, S.: Local rules for protein folding on a triangular lattice and generalized hydrophobicity in the hp model. J. Comput. Biol. 4(3), 275–296 (1997) 3. Agëero-Chapin, G., González-D´ıaz, H., Molina, R., Varona-Santos, J., Uriarte, E., GonzálezD´ıaz, Y.: Novel 2D maps and coupling numbers for protein sequences. The first QSAR study of polygalacturonases; isolation and prediction of a novel sequence from psidium guajava L. FEBS Lett. 580, 723–730 (2006) 4. Agrawal, V.K., Banerji, M., Gupta, M., Singh, J., Khadikar, P.V., Supuran, C.T.: QSAR study on carbonic anhydrase inhibitors: water-soluble sulfonamides incorporating beta-alanyl moieties, possessing long lasting-intra ocular pressure lowering properties – a molecular connectivity approach. Eur. J. Med. Chem. 40(10), 1002–1012 (2005) 5. Aguero-Chapin, G., González-D´ıaz, H., de la Riva, G., Rodriguez, E., Sanchez-Rodriguez, A., Podda, G., Vazquez-Padron, R.I.: MMM-QSAR recognition of ribonucleases without alignment: comparison with an HMM model and isolation from schizosaccharomyces pombe, prediction, and experimental assay of a new sequence. J. Chem. Inf. Model. 48(2), 434–448 (2008) 6. Albert, R., Barabsi, A.L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74(1), 4797 (2002) 7. Alejandro S.-F., Ernesto P.-R., Lorenzo S.: Protein homology detection and fold inference through multiple alignment entropy profiles (2008) 8. Anderson, P.G., Johansson, F., Tanner, D.: Enantioselective addition of organolithium reagents to imines mediated by C2-symmetric bis(aziridine) ligands. Tetrahedron 54, 11549– 11566 (1998) 9. Antunes, A., Ramos, M.J.: Discovery of a large number of previously unrecognized mitochondrial pseudogenes in fish genomes. Genomics 86(6), 708–717 (2005) 10. Ariëns, E.J., Soudijn, W., Timmermans, P.B.M.W.M.: Stereochemistry and Biological Activity of Drugs. Blackwell Scientific, Oxford (1983) 11. Ramos de Armas, R., González-D´ıaz, H., Molina, R., Uriarte, E.: Markovian backbone negentropies: Molecular descriptors for protein research. i. predicting protein stability in arc repressor mutants. Proteins 56(4), 715–723 (2004) 12. Arrasate, S., Lete, E., Sotomayor, N.: Synthesis of enantiomerically enriched amines by chiral ligand mediated addition of organolithium reagents to imines. Tetrahedron Asymmetry 12(14), 2077–2082 (2001) 13. Arteca, G.A., Tapia, O.: Characterization of fold diversity among proteins with the same number of amino acid residues. J. Chem. Inf. Comput. Sci. 39(4), 642–649 (1999) 14. Balaban, A.T., Basak, S.C., Beteringhe, A., Mills, D., Supuran, C.T.: QSAR study using topological indices for inhibition of carbonic anhydrase ii by sulfanilamides and schiff bases. Mol. Divers 8(4), 401–412 (2004) 15. Barabasi, A.L., Bonabeau, E.: Scale-free networks. Sci. Am. 288(5), 60–69 (2003) 16. Basketter, D., Scholes, E.: Comparison of the local lymph node assay with the guinea-pig maximization test for the detection of a range of contact allergens. Fd. Chem. Toxic. 30, 65–69 (1992) 17. Batagelj, V., Mrvar, A.: Pajek 1.15 (2006) 18. Batista, J., Godden, J.W., Bajorath, J.: Assessment of molecular similrity from the analysis if randomly generated structural fragment populations. J. Chem. Inf. Model. 46 (2006) 19. Berger, B., Leighton, T.: Protein folding in the hydrophobic-hydrophilic (hp) model is npcomplete. J. Comput. Biol. 5(1), 27–40 (1998) 20. Berman, H., Henrick, K., Nakamura, H.: Announcing the worldwide protein data bank. Nat. Struct. Biol. 10(12), 980 (2003)

250


21. Berrisford, D.J.: Catalytic asymmetric C–C bond formation: new enolato- and organolithium chemistry. Angew. Chem., Int. Ed. Engl. 34, 178–180 (1995) 22. Bloch, R.: Additions of organometallic reagents to CDN bonds: reactivity and selectivity. Chem. Rev. 98, 1404–1438 (1998) 23. Bonchev, D.: Information Theoretic Indices for Characterization of Chemical Structures. Research Studies Press, Chichester (1983) 24. Bonchev, D.: Complexity in Chemistry, Biology, and Ecology. Springer, New York (2005) 25. Bonchev, D., Buck, G.A.: From molecular to biological structure and back. J. Chem. Inf. Model. 47(3), 909–917 (2007) 26. Bork, P., Jensen, L.J., von Mering, C., Ramani, A.K., Lee, I., Marcotte, E.M.: Protein interaction networks from yeast to human. Curr. Opin. Struct. Biol. 14(3), 292–299 (2004) 27. Bornholdt, S., Schuster, H.: Handbook of Graphs and Complex Networks: From the Genome to the Internet. Wiley-VCH GmbH & CO. KGa., Wheinheim (2003) 28. Breiger, R.: The analysis of social networks. In: Hardy, M., et al. (eds.) Handbook of Data Analysis, pp. 505–526. Sage Publications, London (2004) 29. Buswell, C.M., Herlihy, Y.M., Lawrence, L.M., McGuiggan, J.T., Marsh, P.D., Keevil, C.W., Leach, S.A.: Extended survival and persistence of campylobacter spp. in water and aquatic biofilms and their detection by immunofluorescent-antibody and -rRNA staining. Appl. Environ. Microbiol. 64(2), 733–741 (1998) 30. Cabello, N., Kizirian, J.C., Alexakis, A.: Enantioselective addition of aryllithium reagents to aromatic imines mediated by 1,2-diamine ligands. Tetrahedron Lett. 45, 4639–4642 (2004) 31. Cabello, N., Kizirian, J.C., Gille, S., Alexakis, A., Bernardinelli, G., Pinchard, L., Caille, J.C.: Simple 1,2-diamine ligands for asymmetric addition of aryllithium reagents to imines. Eur. J. Org. Chem. 4835–4842 (2005) 32. Casanola-Martin, G.M., Marrero-Ponce, Y., Khan, M.T., Ather, A., Khan, K.M., Torrens, F., Rotondo, R.: Dragon method for finding novel tyrosinase inhibitors: Biosilico identification and experimental in vitro assays. Eur. J. Med. Chem. 42(11-12), 1370–1381 (2007) 33. Castillo-Garit, J.A., Marrero-Ponce, Y., Torrens, F., Garcia-Domenech, R., Romero-Zaldivar, V.: Bond-based 3d-chiral linear indices: theory and qsar applications to central chirality codification. J. Comput. Chem. 29(15), 2500–2512 (2008) 34. Chen, M., Huang, W.Q.: A branch and bound algorithm for the protein folding problem in the hp lattice model. Genom. Proteomics Bioinformatics 3(4), 225–230 (2005) 35. Cheng, Z., Ren, J., Li, Y., Chang, W., Chen, Z.: Study on the multiple mechanisms underlying the reaction between hydroxyl radical and phenolic compounds by qualitative structure and activity relationship. Bioorg. Med. Chem. 10(12), 4067–4073 (2002) 36. Chou, K.C., Wei, D.Q., Du, Q.S., Sirois, S., Zhong, W.Z.: Review: Progress in computational approach to drug development against sars. Curr. Med. Chem. 13, 3263–3270 (2006) 37. Costerton, J.W., Geesey, G.G., Cheng, K.J.: How bacteria stick. Sci. Am. 238(1), 86–95 (1978) 38. Craig, C.: Social structure. Dictionary of the Social Sciences. Oxford University Press, Oxford (2002) 39. Critchlow, D., Shuying, L., Nourijelyani, K., Pearl, D.: Some statistical methods for phylogenetic trees with application to HIV disease. Math. Comput. Model. 32(1-2), 69–81 (2000) 40. Cruz-Monteagudo, M., González-D´ıaz, H.: Unified drug-target interaction thermodynamic markov model using stochastic entropies to predict multiple drugs side effects. Eur. J. Med. Chem. 40(10), 1030–1041 (2005) 41. Cruz-Monteagudo, M., González-D´ıaz, H., Agero-Chapin, G., Santana, L., Borges, F., Domnguez, R.E., Podda, G., Uriarte, E.: Computational chemistry development of a unified free energy markov model for the distribution of 1300 chemicals to 38 different environmental or biological systems. J. Comput. Chem. 28, 1909–1922 (2007a) 42. Cruz-Monteagudo, M., González-D´ıaz, H., Aguero-Chapin, G., Santana, L., Borges, F., Dominguez, E.R., Podda, G., Uriarte, E.: Computational chemistry development of a unified free energy markov model for the distribution of 1300 chemicals to 38 different environmental or biological systems. J. Comput. Chem. 28(11), 1909–1923 (2007b)


251

43. Cruz-Monteagudo, M., González-D´ıaz, H., Borges, F., Dominguez, E., Cordeiro, M.N.: 3D-mednes: An alternative “in silico” technique for chemical research in toxicology. 2. quantitative proteome-toxicity relationships (qptr) based on mass spectrum spiral entropy. Chem. Res. Toxicol. (21), 619–632 (2008a) 44. Cruz-Monteagudo, M., Munteanu, C., Borges, F., Cordeiro, M., Uriarte, E., Chou, K.C., González-D´ıaz, H.: Stochastic molecular descriptors for polymers. 4. study of complex mixtures with topological indices of mass spectra spiral and star networks: The blood proteome case. Polymer 49(25), 5575–5587 (2008c) 45. Cruz-Monteagudo, M., Munteanu, C.R., Borges, F., Cordeiro, M.N., Uriarte, E., GonzálezD´ıaz, H.: Quantitative proteome-property relationships (qpprs). part 1: finding biomarkers of organic drugs with mean markov connectivity indices of spiral networks of blood mass spectra. Bioorg. Med. Chem. 16(22), 9684–9693 (2008b) 46. Das, B., Meirovitch, H.: Solvation parameters for predicting the structure of surface loops in proteins: transferability and entropic effects. Proteins 51(3), 470–483 (2003) 47. De, P., Singh, A.E., Wong, T., Yacoub, W., Jolly, A.M.: Sexual network analysis of a gonorrhoea outbreak. Sex. Transm. Infect. 80(4), 280–285 (2004) 48. Dea-Ayuela, M.A., Perez-Castillo, Y., Meneses-Marcel, A., Ubeira, F.M., Bolas-Fernandez, F., Chou, K.C., González-D´ıaz, H.: Hp-lattice qsar for dynein proteins: experimental proteomics (2d-electrophoresis, mass spectrometry) and theoretic study of a leishmania infantum sequence. Bioorg. Med. Chem. 16(16), 7770–7776 (2008) 49. Dehmer, M.: A novel method for measuring the structural information content of networks. Cybern. Syst. 39(8), 825–842 (2008) 50. Dehmer, M.: Information-theoretic concepts for the analysis of complex networks. Appl. Artif. Intell. 22(7 & 8), 684–706 (2008) 51. Denmark, S.E., Nakajima, N., Nicaise, O.J.C.: Asymmetric addition of organolithium reagents to imines. J. Am. Chem. Soc. 116, 8797–8798 (1994) 52. Denmark, S.E., Nicaise, O.J.C.: Ligand-mediated addition of organometallic reagents to azomethine functions. Chem. Commun. 999–1004 (1996) 53. Denmark, S.E., Nicaise, O.J.C.: In: Jacobsen, E.N., Pfaltz, A., Yamamoto, H. (eds.) Comprehensive Asymmetric Catalysis, vol. II, 921–961. Springer, Berlin (1999) 54. Denmark, S.E., Stiff, C.M.: Effect of ligand structure in the bisoxazoline mediated asymmetric addition of methyllithium to imines. J. Organic Chem. 65, 5875–5878 (2000) 55. Devah, P.: The mark of a criminal record. Am. J. Soc. 108, 937–975 (2003) 56. Devillers, J., Balaban, A.T.: Topological Indices and Related Descriptors in QSAR and QSPR. Gordon and Breach, The Netherlands (1999) 57. Dobson, P.D., Cai, Y.D., Stapley, B.J., Doig, A.J.: Prediction of protein function in the absence of significant sequence similarity. Curr. Med. Chem. 11(16), 2135–2142 (2004) 58. Eliel, E.L., Wilen, S.H., Mander, L.N.: Stereochemistry of Organic Compounds. Wiley, New York (1994) 59. Elvers, K.T., Leeming, K., Moore, C.P., Lappin-Scott, H.M.: Bacterial-fungal biofilms in flowing water photo-processing tanks. J. Appl. Microbiol. 84(4), 607–618 (1998) 60. Enders, D., Reinhold, U.: Asymmetric synthesis of amines by nucleophilic 1,2-addition of organometallic reagents to the CN-double bond. Tetrahedron Asymmetry 8, 1895–1946 (1997) ´ 61. Erhan, D., LHeureux P.J., Yue, S.Y., Bengio, Y.: Collaborative filtering on a family of biological targets. J. Chem. Inf. Model. 46(2), 626–635 (2006) 62. Estrada, E.: Characterization of the folding degree of proteins. Bioinformatics 18, 697–704 (2002) 63. Estrada, E.: Virtual identification of essential proteins within the protein interaction network of yeast. Proteomics 6(1), 35–40 (2006) 64. Estrada, E.: Food webs robustness to biodiversity loss: the roles of connectance, expansibility and degree distribution. J. Theor. Biol. 244(2), 296–307 (2007) 65. Estrada, E., Molina, E.: 3D connectivity indices in QSPR/QSAR studies. J. Chem. Inf. Comput. Sci. 41(3), 791–797 (2001)

252


66. Estrada, E., Uriarte, E.: Recent advances on the role of topological indices in drug discovery research. Curr. Med. Chem. 8, 1573–1588 (2001) 67. Estrada, E., Uriarte, E., Vilar, S.: Effect of protein backbone folding on the stability of proteinligand complexes. J. Proteome. Res. 5, 105–111 (2006) 68. Evans, P.: Contact and respiratory allergy; a regulatory perspective. In: Seiler, J., Autrup, J., Autrup, H. (eds.) Diversification in Toxicology – Man and Environment, pp. 275–284. Springer, Berlin (1998) 69. Ferino, G., González-D´ıaz, H., Delogu, G., Podda, G., Uriarte, E.: Using spectral moments of spiral networks based on PSA/MASS spectra outcomes to derive quantitative proteome-disease relationships (qpdrs) and predicting prostate cancer. Biochem. Biophys. Res. Commun. 372(2), 320–325 (2008) 70. Forst, C.V.: Host-pathogen systems biology. DDT 11(5-6), 220–227 (2006) 71. Fowler, J.H., Jeon, S.: The authority of supreme court precedent. Soc. Network 30, 16–30 (2008) 72. Garcia-Domenech, R., Galvez, J., de Julian-Ortiz, J.V., Pogliani, L.: Some new trends in chemical graph theory. Chem. Rev. 108(3), 1127–1169 (2008) 73. Garcia-Garcia, A., Galvez, J., de Julian-Ortiz, J.V., Garcia-Domenech, R., Munoz, C., Guna, R., Borras, R.: Search of chemical scaffolds for novel antituberculosis agents. J. Biomol. Screen. 10(3), 206–214 (2005) 74. Gates, M.A.: A simple way to look at DNA. J. Theor. Biol. 119, 319–328 (1986) 75. Gertz, J., Elfond, G., Shustrova, A., Weisinger, M., Pellegrini, M., Cokus, S., Rothschild, B.: Inferring protein interactions from phylogenetic distance matrices. Bioinformatics 19(16), 2039–2045 (2003) 76. Gille, S., Cabello, N., Kizirian, J.C., Alexakis, A.: A new pseudo c2-symmetric tertiary diamine for the enantioselective addition of meli to aromatic imines. Tetrahedron Asymmetry 17, 1045–1047 (2006) 77. Goh, C.S., Cohen, F.E.: Co-evolutionary analysis reveals insights into protein–protein interactions. J. Mol. Biol. 324(1), 177–192 (2002) 78. Gonzalez, M.P., Moldes del Carmen Teran, M.: A tops-mode approach to predict adenosine kinase inhibition. Bioorg. Med. Chem. Lett. 14(12), 3077–3079 (2004c) 79. Gonzalez, M.P., del Carmen Teran Moldes, M.: A tops-mode approach to predict affinity for a1 adenosine receptors. 2-(arylamino)adenosine analogues. Bioorg. Med. Chem. 12(11), 2985–2993 (2004a) 80. Gonzalez, M.P., Diaz, H.G., Cabrera, M.A., Ruiz, R.M.: A novel approach to predict a toxicological property of aromatic compounds in the tetrahymena pyriformis. Bioorg. Med. Chem. 12(4), 735–744 (2004b) 81. González-D´ıaz, H., Agäero-Chapin, G., Varona, J., Molina, R., Delogu, G., Santana, L., Uriarte, E., Gianni, P.: 2D-RNA-coupling numbers: A new computational chemistry approach to link secondary structuretopology with biological function. J. Comput. Chem. 28, 1049– 1056 (2007e) 82. González-D´ıaz, H., Aguero-Chapin, G., Varona-Santos, J., Molina, R., de la Riva, G., Uriarte, E.: 2d rna-qsar: assigning acc oxidase family membership with stochastic molecular descriptors; isolation and prediction of a sequence from psidium guajava L. Bioorg. Med. Chem. Lett. 15(11), 2932–2937 (2005a) 83. González-D´ıaz, H., González-D´ıaz, Y., Santana, L., Ubeira, F.M., Uriarte, E.: Proteomics, networks and connectivity indices. Proteomics 8, 750–778 (2008a) 84. González-D´ıaz, H., Marrero, Y., Hernandez, I., Bastida, I., Tenorio, E., Nasco, O., Uriarte, E., Castanedo, N., Cabrera, M.A., Aguila, E., Marrero, O., Morales, A., Perez, M.: 3Dmednes: an alternative “in silico” technique for chemical research in toxicology. 1. prediction of chemically induced agranulocytosis. Chem. Res. Toxicol. 16(10), 1318–1327 (2003) 85. González-D´ıaz, H., Molina, R., Uriarte, E.: Markov entropy backbone electrostatic descriptors for predicting proteins biological activity. Bioorg. Med. Chem. Lett. 14(18), 4691–4695 (2004)


253

86. González-D´ıaz, H., Molina, R., Uriarte, E.: Stochastic molecular descriptors for polymers 1. modeling the properties of icosahedral viruses with 3d-markovian negentropies. Polymer 45(11), 3845–3853 (2004) 87. González-D´ıaz, H., Molina, R., Uriarte, E.: Recognition of stable protein mutants with 3d stochastic average electrostatic potentials. FEBS Lett. 579(20), 4297–4301 (2005) 88. González-D´ıaz, H., Molina, R., Uriarte, E.: Recognition of stable protein mutants with 3d stochastic average electrostatic potentials. FEBS Lett. 579(20), 4297–4301 (2005b) 89. González-D´ıaz, H., Molina-Ruiz, R., Hernandez, I.: March-inside v3.0 (markov chains invariants for simulation & design); windows supported version under request to the main author contact email: [email protected] (2007c) 90. González-D´ıaz, H., Pérez-Bello, A., Cruz-Monteagudo, M., González-D´ıaz, Y., Santana, L., Uriarte, E.: Chemometrics for qsar with low sequence homology: Mycobacterial promoter sequences recognition with 2d-rna entropies. Chemom. Intell. Lab. Systs. 85, 20–26 (2007b) 91. González-D´ıaz, H., Perez-Castillo, Y., Podda, G., Uriarte, E.: Computational chemistry comparison of stable/nonstable protein mutants classification models based on 3d and topological indices. J. Comput. Chem. 28(12), 1990–1995 (2007c) 92. González-D´ıaz, H., Prado-Prado, F.: Unified qsar and network-based computational chemistry approach to antimicrobials, part 1: Multispecies activity models for antifungals. J. Comput. Chem. 29, 656–657 (2008) 93. González-D´ıaz, H., Prado-Prado, F., Ubeira, F.M.: Predicting antimicrobial drugs and targets with the march-inside approach. Curr. Top. Med. Chem. 8(18), 1676–1690 (2008) 94. González-D´ıaz, H., Prado-Prado, F.J., Santana, L., Uriarte, E.: Unify qsar approach to antimicrobials. part 1: Predicting antifungal activity against different species. Bioorg. Med. Chem. 14, 5973–5980 (2006a) 95. González-D´ıaz, H., Saiz-Urra, L., Molina, R., Santana, L., Uriarte, E.: A model for the recognition of protein kinases based on the entropy of 3d van der waals interactions. J. Proteome. Res. 6(2), 904–908 (2007d) 96. González-D´ıaz, H., Saiz-Urra, L., Molina, R., Santana, L., Uriarte, E.: A model for the recognition of protein kinases based on the entropy of 3d van der waals interactions. J. Proteome. Res. 6(2), 904–908 (2007e) 97. González-D´ıaz, H., Sa´ız-Urra, L., Molina, R., Uriarte, E.: Stochastic molecular descriptors for polymers. 2. spherical truncation of electrostatic interactions on entropy based polymers 3d-qsar. Polymer 46, 2791–2798 (2005b) 98. González-D´ıaz, H., Sanchez-Gonzalez, A., González-D´ıaz, Y.: 3d-qsar study for DNA cleavage proteins with a potential anti-tumor atcun-like motif. J. Inorg. Biochem. 100(7), 1290–1297 (2006) 99. González-D´ıaz, H., Uriarte, E.: Biopolymer stochastic moments. i. modeling human rhinovirus cellular recognition with protein surface electrostatic moments. Biopolymers 77(5), 296–303 (2005a) 100. González-D´ıaz, H., Uriarte, E.: Proteins qsar with markov average electrostatic potentials. Bioorg. Med. Chem. Lett 15(22), 5088–5094 (2005b) 101. González-D´ıaz, H., Uriarte, E., Ramos de Armas, R.: Predicting stability of arc repressor mutants with protein stochastic moments. Bioorg. Med. Chem. 13(2), 323–331 (2005c) 102. González-D´ıaz, H., Vilar, S., Santana, L., Uriarte, E.: Medicinal chemistry and bioinformatics current trends in drugs discovery with networks topological indices. Curr. Top. Med. Chem. 7(10), 1025–1039 (2007a) 103. González-D´ıaz, H., Vina, D., Santana, L., de Clercq, E., Uriarte, E.: Stochastic entropy qsar for the in silico discovery of anticancer compounds: prediction, synthesis, and in vitro assay of new purine carbanucleosides. Bioorg. Med. Chem. 14(4), 1095–1107 (2006b) 104. Graham, D.J.: Information content and organic molecules: Aggregation states and solvent effects. J. Chem. Inf. Model. 45(1223) (2005) 105. Graham, D.J.: Information content in organic molecules: Brownian processing at low levels. J. Chem. Inf. Model. 47(2), 376–389 (2007)

254


106. Graham, D.J., Kim, M.: Information and classical thermodynamic transformations. J. Phys. Chem. B 112, 10585–10593 (2008) 107. Gupta, A., Manuch, J., Stacho, L.: Inverse protein folding in 2d hp mode (extended abstract). Proc IEEE Comput. Syst. Bioinform. Conf. 311–318 (2004) 108. Gupta, A., Manuch, J., Stacho, L.: Structure-approximating inverse protein folding problem in the 2d hp model. J. Comput. Biol. 12(10), 1328–1345 (2005) 109. Hamacher, K.: Information theoretical measures to analyze trajectories in rational molecular design. J. Comput. Chem. 28(16), 2576–2580 (2007) 110. Hampl, V., Cepicka, I., Flegr, J., Tachezy, J., Kulda, J.: Critical analysis of the topology and rooting of the parabasalian 16s rRNA tree. Mol. Phylogenet. Evol. 32(3), 711–723 (2004) 111. Han, L., Cui, J., Lin, H., Ji, Z., Cao, Z., Li, Y., Chen, Y.: Recent progresses in the application of machine learning approach for predicting protein functional class independent of sequence similarity. Proteomics 6, 4023–4037 (2006) 112. Harford, C., Sarkar, B.: Neuromedin C binds Cu(II) and Ni(II) via the atcun motif: implications for the CNS and cancer growth. Biochem. Biophys. Res. Commun. 209(3), 877– 882 (1995) 113. Harvey, P., Pagel, M.: The Comparative Method in Evolutionary Biology, Ecology and Evolution, vol. 1. Oxford University Press, Oxford (1991) 114. Hasegawa, M., Taniyama, D., Tomioka, K.: Facile asymmetric synthesis of a-amino acids employing chiral ligand-mediated asymmetric addition reactions of phenyllithium with imines. Tetrahedron 56, 10153–10158 (2000) 115. Hjelle, B., Jenison, S., Torrez-Martinez, N., Yamada, T., Nolte, K., Zumwalt, R., MacInnes, K., Myers, G.: A novel hantavirus associated with an outbreak of fatal respiratory disease in the southwestern united states: evolutionary relationships to known hantaviruses. J. Virol. 68(2), 592–596 (1994) 116. Hua, S., Sun, Z.: Support vector machine approach for protein subcellular localization prediction. Bioinformatics 17(8), 721–728 (2001) 117. Inc, H.: Hyperchem release 7.0.3 (2002) 118. Inoue, I., Shindo, M., Koga, K., Kanai, M., Tomioka, K.: Enantioselective reaction of an imine with methyllithium catalyzed by a chiral ligand. Tetrahedron Asymmetry 6, 2527–2533 (1995) 119. Inoue, I., Shindo, M., Koga, K., Tomioka, K.: Asymmetric 1,2-addition of organolithium to aldimines catalyzed by chiral ligand. Tetrahedron 50, 4429–4438 (1994) 120. Jacques, J., Collet, A., Wilen, S.H.: Enantiomers, Racemates, and Resolution. Wiley, New York (1981) 121. Jefferson, K.K.: What drives bacteria to produce a biofilm? FEMS Microbiol. Lett. 236(2), 163–173 (2004) 122. Jensen, S.T., Shen, L., Liu, J.S.: Combining phylogenetic motif discovery and motif clustering to predict co-regulated genes. Bioinformatics 21(20), 3832–3839 (2005) 123. Jeong, H., Tombor, B., Albert, R., Oltvai, Z.N., Barabasi, A.L.: The large-scale organization of metabolic networks. Nature 407(6804), 651–654 (2000) 124. Jiang, M., Zhu, B.: Protein folding on the hexagonal lattice in the hp model. J. Bioinform. Comput. Biol. 3(1), 19–34 (2005) 125. Johnson, J.C., Orbach, M.K.: Perceiving the political landscape: ego biases in cognitive political networks. Soc. Network 24, 291–310 (2002) 126. Junker, B.H., Koschuetzki, D., Schreiber, F.: Exploration of biological network centralities with centibin. BMC Bioinformatics 7(1), 219 (2006) 127. Karelson, M.: Molecular Descriptors in QSAR/QSPR. Wiley-Interscience, New York (2000) 128. Karplus, M., McCammon, J.A.: Molecular dynamics simulations of biomolecules. Nat. Struct. Biol. 9(9), 646–652 (2002) 129. Kier, L.B.: Use of molecular negentropy to encode structure governing biological activity. J. Pharm. Sci. 69(7), 807–810 (1980) 130. Kimber, I., Hilton, J., Botham, P.: Identification of contact allergens using the murine local lymph node assay. comparisons with the buehler occluded patch test in guinea pigs. J. Appl. Toxicol. 10, 173–180 (1990)


255

131. Kimoto, E., Tanaka, H., Gyotoku, J., Morishige, F., Pauling, L.: Enhancement of antitumor activity of ascorbate against ehrlich ascites tumor cells by the copper:glycylglycylhistidine complex. Cancer Res. 43(2), 824–828 (1983) 132. Kizirian, J.C., Cabello, N., Pinchard, L., Caille, J.C., Alexakis, A.: Enantioselective addition of methyllithium to aromatic imines catalyzed by c2 symmetric tertiary diamines. Tetrahedron 61, 8939–8946 (2005) 133. Klein, J.: The chemistry. In: Patai, S. (ed.) The Chemistry of Double-bonded Functional Groups: Suppement A, vol. 2. Wiley, Chichester (1989) 134. Kleinman, E.F., Volkmann, R.A.: In: Heathcock, C.H. (ed.) Comprehensive Organic Synthesis, Additions to C-X p-Bonds, Part 2, vol. 2. Pergamon Press, Oxford (1991) 135. Klovdahl, A.S., Potterat, J.J., Woodhouse, D.E., Muth, J.B., Muth, S.Q., Darrow, W.W.: Social networks and infectious disease: The colorado springs study. Soc. Sci. Med. 38, 79–88 (1994) 136. Kolenbrander, P., Andersen, R., Clemans, D., Whittaker, C., Klier, C.: Potential role of functionally similar coaggregation mediators in bacterial succession. In: Newman, H., et al. (eds.) Dental Plaque Revisited: Oral Biofilms in Health and Disease, pp. 171–186. Bioline Press, Cardiff (1999) 137. Koschtzki, D.: Centibin, Centralities in Biological Networks. IPK Gatersleben, Germany (2004) 138. Kowalski, R.B., Wold, S.: Pattern recognition in chemistry. In: Krishnaiah, R.P., et al. (eds.) Handbook of Statistic, pp. 673–697. North Holland Publishing, Amsterdam (1982) 139. Kutner, M., Nachtsheim, C., Neter, J., Li, W.: Standardized multiple regression model. In: Applied Linear Statistical Models, 5th edn., pp. 271–277. McGraw Hill, New York (2005) 140. Lazareva-Ulitsky, B., Diemer, K., Thomas, P.D.: On the quality of tree-based protein classification. Bioinformatics 21(9), 1876–1890 (2005) 141. Leong, P.M., Morgenthaler, S.: Random walk and gap plots of dna sequences. Comput. Appl. Biosci. 11, 503–507 (1995) 142. Liao, B., Luo, J., Li, R., Zhu, W.: RNA secondary structure 2d graphical representation without degeneracy. Int. J. Quant. Chem. 106(8), 1749–1755 (2006) 143. Liljeros, F., Edling, C.R., Amaral, L.A.N., Stanley, H.E., Aberg, Y.: The webof human sexual contacts. Nature 411, 907–908 (2001) 144. Lorand, T., Kocsis, B., Sohar, P., Nagy, G., Jozsef, P., Kispal, G., Laszlo, R., Prokai, L.: Synthesis and antibacterial activity of fused mannich ketones. Eur. J. Med. Chem. 37(10), 803–812 (2002) 145. Mace, R., Holden, C.J.: A phylogenetic approach to cultural evolution. Trends Ecol. Evol. 20(3), 116–121 (2005) 146. Marques, A.T., Antunes, A., Fernandes, P.A., Ramos, M.J.: Comparative evolutionary genomics of the hadh2 gene encoding abeta-binding alcohol dehydrogenase/17betahydroxysteroid dehydrogenase type 10 (abad/hsd10). BMC Genom. 7, 202 (2006) 147. Marrero-Ponce, Y., Medina-Marrero, R., Castillo-Garit, J.A., Romero-Zaldivar, V., Torrens, F., Castro, E.A.: Protein linear indices of the ’macromolecular pseudograph alpha-carbon atom adjacency matrix’ in bioinformatics. part 1: prediction of protein stability effects of a complete set of alanine substitutions in arc repressor. Bioorg. Med. Chem. 13(8), 3003–3015 (2005) 148. Marrero-Ponce, Y., Nodarse, D., González-D´ıaz, H., Ramos de Armas, R., RomeroZaldivar, V., Torrens, F., Castro, E.A.: Nucleic acid quadratic indices of the “macromolecular graphs nucleotides adjacency matrix”. modeling of footprints after the interaction of paromomycin with the hiv-1?-RNA packaging region. Int. J. Mol. Sci. 5, 276–293 (2004) 149. Maslovat, D., Chus, R., Lee, T.D., Franks, I.M.: Contextual interference: single task versus multi-task learning. Mot. Contr. 8(2), 213–233 (2004) 150. Mason, O., Verwoerd, M.: Graph theory and networks in biology. IET Syst. Biol. 1(2), 89–119 (2007) 151. McCammon, J.A., Gelin, B.R., Karplus, M.: Dynamics of folded proteins. Nature 267(5612), 585–590 (1977)

256


152. McCammon, J.A., Karplus, M.: Internal motions of antibody molecules. Nature 268(5622), 765–766 (1977) 153. Microsoft.Corp.: Microsoft excel (2002) 154. Milla, M.E., Brown, B.M., Sauer, R.T.: Protein stability effects of a complete set of alanine substitutions in arc repressor. Nat. Struct. Biol. 1(8), 518–523 (1994) 155. Mizruchi, M.S.: The American Corporate Network , 1904–1974. Sage, Beverly Hills (1982) 156. Molina, E., Diaz, H.G., Gonzalez, M.P., Rodriguez, E., Uriarte, E.: Designing antibacterial compounds through a topological substructural approach. J. Chem. Inf. Comput. Sci. 44(2), 515–521 (2004) 157. Moreno, J.L.: Who Shall Survive? Beacon House, New York (1934) 158. Moser, H., Rihs, G., Santer, H.Z.: Naturforsch 37B, 451–462 (1982) 159. Mukhopadhyay, A., Peterson, R.T.: Fishing for new antimicrobials. Curr. Opin. Chem. Biol. 10(4), 327–333 (2006) 160. Munteanu, C.R., González-D´ıaz, H., Magalhaes, A.L.: Enzymes/non-enzymes classification model complexity based on composition, sequence, 3d and topological indices. J. Theor. Biol. 254(2), 476–482 (2008) 161. Munteanu, C.R., Gonzles-Diz, H.: S2snet - sequence to star network, reg. no. 03/2008/1338 (2008) 162. Nalewajski, R.F.: Information Theory of Molecular Systems. Elsevier, Amsterdam (2006) 163. Nandy, A.: Two-dimensional graphical representation of dna sequences and intron-exon discrimination in intron-rich sequences. Comput. Appl. Biosci. 12(1), 55–62 (1996) 164. Newman, M.: The structure and function of complex networks. SIAM Rev. 56, 167–256 (2003) 165. North, M.: Contemp. Org. Synth. 3, 323–343 (1996) 166. Padgett, J.F., Ansell, C.K.J.F.: Robust action and the rise of the medici, 1400–1434. Am. J. Sociol. 98, 259–1319 (1993) 167. Patlewicz, G.Y., Basketter, D.A., Pease, C.K., Wilson, K., Wright, Z.M., Roberts, D.W., Bernard, G., Arnau, E.G., Lepoittevin, J.P.: Further evaluation of quantitative structure– activity relationship models for the prediction of the skin sensitization potency of selected fragrance allergens. Contact Dermatitis 50(2), 91–97 (2004) 168. Perez-Bello, A., Munteanu, C.R., Ubeira, F.M., Lopes De Magalhaes, A., Uriarte, E., González-D´ıaz, H.: Alignment-free prediction of mycobacterial dna promoters based on pseudo-folding lattice network or star-graph topological indices. J. Theor. Biol. (2008) 169. Perez Gonzalez, M., González-D´ıaz, H., Molina Ruiz, R., Cabrera, M.A., Ramos de Armas, R.: Tops-mode based qsars derived from heterogeneous series of compounds. applications to the design of new herbicides. J. Chem. Inf. Comput. Sci. 43(4), 1192–1199 (2003) 170. Perron, Q., Alexakis, A.: Synthesis and application of a new pseudo c2-symmetric tertiary diamine for the enantioselective addition of meli to aromatic imines. Tetrahedron Asymmetry 18, 2503–2506 (2007) 171. Pompe, M., Veber, M., Randic, M., Balaban, A.T.: Using variable and fixed topological indices for the prediction of reaction rate constants of volatile unsaturated hydrocarbons with oh radicals. Molecules 9, 1160–1176 (2004) 172. Prado-Prado, F., González-D´ıaz, H., Santana, L., Uriarte, E.: Unified qsar approach to antimicrobials. part 2: Predicting activity against more than 90 different species in order to halt antibacterial resistance. Bioorg. Med. Chem. 15, 897–902 (2007) 173. Prado-Prado, F., González-D´ıaz, H., Martinez de la Vega, O., Ubeira, F.M., Chou, K.C.: Unified qsar approach to antimicrobials. part 3: First multi-tasking qsar model for input-coded prediction, structural back-projection, and complex networks clustering of antiprotozoal compounds. Bioorg. Med. Chem. 16, 5871–5880 (2008) 174. Prado-Prado, F.J., de la Vega, O.M., Uriarte, E., Ubeira, F.M., Chou, K.C., González-D´ıaz, H.: Unified qsar approach to antimicrobials. 4. multi-target qsar modeling and comparative multidistance study of the giant components of antiviral drug-drug complex networks. Bioorg. Med. Chem. 17, 56975 (2009)


257

175. Puslednik, L., Serb, J.M.: Molecular phylogenetics of the pectinidae (mollusca: Bivalvia) and effect of increased taxon sampling and outgroup selection on tree topology. Mol. Phylogenet. Evol. 48(3), 1178–1188 (2008) 176. Ramani, A.K., Marcotte, E.M.: Exploiting the co-evolution of interacting proteins to discover interaction specificity. J. Mol. Biol. 327(1), 273–284 (2003) 177. Randic, M., Balaban, A.T.: On a four-dimensional representation of dna primary sequences. J. Chem. Inf. Comput. Sci. 43(2), 532–539 (2003) 178. Randic, M., Guo, X., Basak, S.C.: On the characterization of dna primary sequences by triplet of nucleic acid bases. J. Chem. Inf. Comput. Sci. 41(3), 619–626 (2001) 179. Ren, Y., Liu, H., Yao, X., Liu, M.: Prediction of ozone tropospheric degradation rate constants by projection pursuit regression. Anal. Chim. Acta 589, 150–158 (2007) 180. Rickard, A.H., Gilbert, P., High, N.J., Kolenbrander, P.E., Handley, P.S.: Bacterial coaggregation: an integral process in the development of multi-species biofilms. Trends Microbiol. 11(2), 94–100 (2003a) 181. Rickard, A.H., McBain, A.J., Ledder, R.G., Handley, P.S., Gilbert, P.: Coaggregation between freshwater bacteria within biofilm and planktonic communities. FEMS Microbiol. Lett. 220(1), 133–140 (2003b) 182. Risch, N., Arend, M.: In: Helmchen, G., Hoffmann, R.W., Mulzer, J., Schaumann, E. (eds.) Methods of Organic Chemistry. Stereoselective Synthesis [Houben-Weyl], Workbench Edition E21, Vol. 3. Thieme, Stuttgart (1996) 183. Rodloff, A.C., Leclercq, R., Debbia, E.A., Canton, R., Oppenheim, B.A., Dowzicky, M.J.: Comparative analysis of antimicrobial susceptibility among organisms from france, germany, italy, spain and the uk as part of the tigecycline evaluation and surveillance trial. Clin. Microbiol. Infect. 14(4), 307–314 (2008) 184. Sabidussi, G.: The centrality index of a graph. Psychometrika 31, 581–603 (1966) 185. Saiz-Urra, L., González-D´ıaz, H., Uriarte, E.: Proteins markovian 3d-qsar with sphericallytruncated average electrostatic potentials. Bioorg. Med. Chem. 13(11), 3641–3647 (2005) 186. Sankararamakrishnan, R., Verma, S., Kumar, S.: Atcun-like metal-binding motifs in proteins: identification and characterization by crystal structure and sequence analysis. Proteins 58(1), 211–221 (2005) 187. Santana, L., Uriarte, E., González-D´ıaz, H., Zagotto, G., Soto-Otero, R., Mendez-Alvarez, E.: A qsar model for in silico screening of mao-a inhibitors. prediction, synthesis, and biological assay of novel coumarins. J. Med. Chem. 49(3), 1149–1156 (2006) 188. Savoia, D.: Progress in the asymmetric synthesis of 1,2-diamines from azomethine compounds. Top. Organomet. Chem. 15, 1–58 (2005) 189. Sciretti, D., Bruscolini, P., Pelizzola, A., Pretti, M., Jaramillo, A.: Computational protein design with side-chain conformational entropy. Proteins 74(1), 176–191 (2008) 190. Seyden-Penne, J.: Chiral Auxiliaries and Ligands in Asymmetric Synthesis. Wiley, New York (1995) 191. SRL, T.: Dragon for windows ver. 5.3, software for molecular descriptor calculations, http://www.talete.mi.it (2005) 192. Stahura, F.L., Godden, J.W., Xue, L., Bajorath, J.: Distinguishing between natural products and synthetic molecules by descriptor shannon entropy analysis and binary qsar calculations. J. Chem. Inf. Comput. Sci. 40(5), 1245–1252 (2000) 193. StatSoft.Inc.: Statistica, Data Analysis Software System, version 6.0 (2002) 194. Stewart, J., Gill, L.: Econometrics, 2nd edn. Prentice Hall, London (1998) 195. Strogatz, S.H.: Exploring complex networks. Nature 410(6825), 268–276 (2001) 196. Taniyama, D., Hasegawa, M., Tomioka, K.: A facile asymmetric synthesis of 1-substituted tetrahydroisoquinoline based on a chiral ligand-mediated addition of organolithium to imine. Tetrahedron Asymmetry 10, 221–223 (1999) 197. Thachuk, C., Shmygelska, A., Hoos, H.H.: A replica exchange monte carlo algorithm for protein folding in the hp model. BMC Bioinform. 8(1), 342 (2007) 198. Tippery, N.P., Les, D.H.: Phylogenetic analysis of the internal transcribed spacer (its) region in menyanthaceae using predicted secondary structure. Mol. Phylogenet. Evol. (2008)

258


199. Todeschini, R., Consonni, V.: Handbook of Molecular Descriptors. Wiley-VCH, New York (2002) 200. Van de Waterbeemd, H.: Discriminant analysis for activity prediction. In: Manhnhold, R., Krogsgaard-Larsen, L., Timmerman, H. (eds.) Methods and Principles in Medicinal Chemistry, Chemometric methods in molecular design, Van Waterbeemd, H. (ed.), vol. 2, 265–288. VCH, Weinhiem (1995a) 201. Van Waterbeemd, H.: Discriminant analysis for activity prediction. In: Van Waterbeemd, H. (ed.) Chemometric methods in molecular design, Method and Principles in Medicinal Chemistry, vol. 2, pp. 265–282. Wiley-VCH, New York (1995b) 202. Van Waterbeemd, H.: Chemometric methods in molecular design, Method and Principles in Medicinal Chemistry, vol. 2. Wiley-VCH, New York (1995c) 203. Vassura, M., Margara, L., Di Lena, P., Medri, F., Fariselli, P., Casadio, R.: Reconstruction of 3d structures from protein contact maps. IEEE/ACM Trans. Comput. Biol. Bioinform. 5(3), 357–367 (2008) 204. Vilar, S., González-D´ıaz, H., Santana, L., Uriarte, E.: Qsar model for alignment-free prediction of human breast cancer biomarkers based on electrostatic potentials of protein pseudofolding hp-lattice networks. J. Comput. Chem. 29, 2613–2622 (2008) 205. Volkmann, R.A.: In: S.L. Schreiber (ed.) Comprehensive Organic Synthesis, Additions to C-X p-Bonds, Part 1, vol. 1. Pergamon Press, Oxford (1991) 206. Volokhov, D.V., Neverov, A.A., George, J., Kong, H., Liu, S.X., Anderson, C., Davidson, M.K., Chizhikov, V.: Genetic analysis of housekeeping genes of members of the genus acholeplasma: phylogeny and complementary molecular markers to the 16s rRNA gene. Mol. Phylogenet. Evol. 44(2), 699–710 (2007) 207. Weka: Waikato Environment for Knowledge Analysis (weka) (2002) 208. Wellman, B., Berkowitz, S.D.: Social Structures: A Network Approach. Cambridge University Press, Cambridge (1988) 209. White Harrison, S.B., Breiger, R.: Social structure from multiple networks: I blockmodels of roles and positions. Am. J. Sociol. 81, 730–780 (1976) 210. Wiener, H.: J. Am. Chem. Soc. 69, 17 (1947) 211. Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. San Francisco (2000) 212. Yamanishi, Y., Araki, M., Gutteridge, A., Honda, W., Kanehisa, M.: Prediction of drug-target interaction networks from the integration of chemical and genomic spaces. Bioinformatics 24(13), i232–i240 (2008) 213. Yildirim, M.A., Goh, K.I., Cusick, M.E., Barabasi, A.L., Vidal, M.: Drug-target network. Nat. Biotechnol. 25(10), 1119–1126 (2007) 214. Zhang, X.S., Wang, Y., Zhan, Z.W., Wu, L.Y., Chen, L.: Exploring protein’s optimal hp configurations by self-organizing mapping. J. Bioinform. Comput. Biol. 3(2), 385–400 (2005) 215. Zhou, H., Zhou, Y.: Stability scale and atomic solvation parameters extracted from 1023 mutation experiments. Proteins 49(4), 483–492 (2002) 216. Zweig, M.H.: Apolipoproteins and lipids in coronary artery disease. analysis of diagnostic accuracy using receiver operating characteristic plots and areas. Arch. Pathol. Lab. Med. 118(2), 141–144 (1994)

Chapter 10

Social Ontologies as Generalized Nearly Acyclic Directed Graphs: A Quantitative Graph Model of Social Tagging Alexander Mehler

Abstract In this paper, we introduce a quantitative graph model of social ontologies as exemplified by the category system of Wikipedia. This is done to contrast structure formation in distributed cognition with classification schemes (by example of the DDC and MeSH), formal ontologies (by example of OpenCyc and SUMO), and terminological ontologies (as exemplified by WordNet). Our basic findings are that social ontologies have a characteristic topology that clearly separates them from other types of ontologies. In this context, we introduce the notion of a Zipfian bipartivity to analyze the relationship of categories and categorized units in distributed cognition. Keywords Generalized nearly acyclic directed graphs • Quantitative network analysis • Social ontology • Wikipedia • Zipfian bipartivity MSC2000 Primary 05C75; Secondary 05C82, 68T50, 90B15, 91D30, 91F20.

1 Introduction In recent years social tagging as part of social bookmarking systems such as del.icio.us [19], BibSonomy [35], or CiteULike [17] has been made a resource for measuring the semantic similarity or relatedness [15] of lexical items [12, 20], of topic labeling [89], and of web content mining [37]. Further, social tagging has been the starting point of building more elaborated network models, for example, in the form of tripartite networks [43]. Beyond that, social tagging has also been

A. Mehler () Faculty of Computer Science and Mathematics, Goethe University Frankfurt am Main, D-60325 Frankfurt am Main, Germany e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 10, © Springer Science+Business Media, LLC 2011

259

260

A. Mehler

made an object of cognitive linguistics [82] and especially of simulation models of language evolution [19, 25]. With the latter approaches, we share the following two starting points: • First, we view meaning constitution as a process of distributed cognition [33], distributed among large groups of interacting agents who collectively generate and structure certain fields of knowledge [82]. Further, unlike mass communication (which realizes a one-to many relation), social tagging constitutes a many-to-many communication in which groups of agents interact to collectively comment, review, or interpret a dynamically growing universe of content units. • Second, social tagging can be characterized in terms of long-term processes of self-organization and short-term processes of self-regulation. Due to the dynamics of the contributions of its participants, social tagging evolves nonrandomly so that it has a law-like structuring. It can be modeled by stochastic laws of the networks spanned by the tags generated in this process where two tags are linked subject to their “co-tagging” of the same content unit. Note that these networks are commonly represented as unipartite but less frequently as tripartite graphs [17, 56]. Social tagging is a process of web-based communication that came into existence with Web 2.0. It is a genuine object of web genre analysis [75] as it did not exist before the advent of the web. While the hypertext types, which have been commonly analyzed in web genre analysis, are instances of mass communication, it is a manyto-many communication which comes into focus through social tagging. In this chapter, we deal with a certain type (of output) of social tagging, namely so-called social ontologies [57]. We do that exclusively by example of the category system of Wikipedia [88]. Social ontologies are related to social tagging in the sense that the former is a product of distributed cognition used by the latter to categorize artefacts on the web [82]. Social ontologies can be characterized as follows: • Typically, social ontologies tend to evolve in an unsupervised, uncontrolled, unplanned, and unmoderated manner.1 They miss an overall architecture or blueprint as they are built by large, heterogeneous communities of interacting agents, henceforth called wikilocutors, who participate in social tagging without explicit mutual negotiation and complete knowledge of the underlying ontology. Obviously, their heterogeneity is manifested by their varying levels of expertise in the fields of knowledge in which they tag. • In this sense, social ontologies are distinguished from terminological ontologies [81] in which the ontological data is primarily generated by a small number of experts who collaborate in a controlled and moderated manner according to a plan of the target ontology. This is in contrast to social ontologies as part of the Wikipedia in which ontological data is generated without distinguishing between expert and nonexpert users. 1

Note that within the social ontologies analyzed here, i.e. Wikipedia category systems, moderation may occur.

10 A Quantitative Graph Model of Social Ontologies

261

• However, social ontologies share with terminological ones the absence of axiomatic definitions which are in turn essential for formal ontologies. Thus, we hypothesize that social ontologies evolve apart from both terminological and formal ontologies as a novel type of purely web-based systems. • Social ontologies share with folksonomies [37] a kind of implicit constitution. However, unlike folksonomies (though like both terminological and formal ontologies), they span hierarchical structures. That is, apart from their lightweight counterpart in the form of folksonomies, social ontologies make explicit the conceptual relations among their constitutive units – to a degree missed in folksonomies. Note that we do not use the notion of a social ontology in terms of philosophy [79]. We may even say that the social ontology as used here is as different from its usage in philosophy as is the general notion of ontology in computer science to that in philosophy. However, we agree with the notion of social ontologies as put forward by [11] in that we also stress the implicitness of the relations among the participants of a social ontology. In this way, a social ontology emerges as a solution to problems of coordination, that is, the sharing of a collectively structured semantic universe. In terms of the typology of [81], we state that the genus proximum of social ontologies are terminological ones with which they share, first, the absence of formal axiomatic definitions and, second, the focus on subtype–supertype or part– whole relations. However, a differentia specifica relates to the way social ontologies are generated in an unmoderated process of implicit cooperation among large numbers of authors. With respect to social ontologies, therefore, we seek law-like characteristics in order to demarcate them from other types of ontologies. That is, unlike many approaches in physics we do not seek universal network characteristics [9] but those which additionally characterize social ontologies by their differences to “nonsocial” ones. We seek a quantitative graph model not of complex networks in general, but of more constrained tree-like structures. In this sense, we also depart from models as exemplified by [5] who describe ontologies in the framework of small-world statistics. Social ontologies are hardly small worlds. Thus, we aim at the laws of structure formation in social ontologies, apart from small-world models, by addressing the following questions: • What kind of structure is spanned by social ontologies? • More formally: What kind of graph class is spanned by social ontologies? • Does this graph model serve as a reliable input to a purely structure-based classification of formal, terminological, and social ontologies, i.e., a classification in terms of their topology? Our main objective is to come up with a graph model of social ontologies that profiles them in terms of quantitative characteristics. We seek fingerprints of their topology which make them identifiable irrespective of their size or age. This is, so to speak, in support of saying that you shall know the type of an ontology by

262

A. Mehler

the topology it spans. In order to implement a classification model in line with this hypothesis we use the following auxiliary concept when speaking about social ontologies: Definition 1. Let S be a social ontology given as an empirical system S D .C; R; >/ such that C is a set of empirical categories and R C 2 is the empirical relation over C where for all .Ci ; Cj / 2 R it holds that Ci is the supertype of its subtype Cj or Ci is a holonym of Cj or Ci is otherwise superordinate to Cj . Further, > 2 C is the unique main category of S. A Social Ontology Graph (SOG) (also called category graph) induced by S is a directed graph D D .V; A; r/ such that there is a surjective homomorphism h W S ! D. In particular, we call h.>/ D r the main category in D. Further, S is called the preimage of D denoted by h1 .D/. Note: D is a qualitative measurement of the social ontology h1 .D/ in the sense that it preserves its structuring (i.e., h is a homomorphism) and that there is no subgraph of D without a pre-image in h1 .D/ (i.e., h is surjective). It is this surjection which guarantees that D does not contain “more structure” than h1 .D/ so that it is a reliable input to measuring its topology. Note finally that a unique main category is always provided by selecting one of the sources of h1 .D/. There already exists a quantitative approach to social ontologies [88]. However, we depart from it as we do not simply enlarge the number of power law statistics of degree distributions. Further, we do not just state that social ontologies are no trees or that they are Directed Acyclic Graphs (DAG) (which, by the way, is wrong). Rather, we extend our focus in two respects: • First, we present measures for locating SOGs more exactly within the continuum of tree-like structures on the one hand and unconstrained graphs on the other. Although come up with a result of doing this, it is a matter of principle that no approach can ever detect this location ultimately as we deal with natural systems of an unknown complexity. • Second, we do not just attribute the membership of SOGs to some graph classes but classify them automatically in relation to ontologies of different types. There are at least two application areas for this kind of research: • Web genres: Social ontologies are a newly emerging genre on the web, i.e., a generic web genre. From that perspective, social ontologies form a genuine object of research by themselves. • Information processing: Social ontologies are an invaluable source of terminological knowledge in web mining [21,45,58,69,84,89]. They provide large-scale and flexible knowledge systems for building open content models. An open content model is a topic-related classification model in which the content categories are not enumerated in advance (as in supervised learning) or the result of labeling clusters found in a given set of empirical data (as in unsupervised learning). Rather, open content models explore their topic labels from an open, that is, ever-growing social ontology. In a nutshell: an open content model is a


263

model of topic-related classification which gets its topic model from a social ontology with which it co-evolves. For such an approach, the present paper provides an empirical account of what kind of structures have to be expected when dealing with social ontologies. More specifically, which of their structural aspects are law-like and which are not. Knowledge about this is needed to develop algorithms for assessing the quality or genuineness of the branches of a SOG as resources for classifying textual input. Note that we do not argue that social ontologies are worse than formal or terminological ontologies due to their structural peculiarities or novelty. For an example of a study of information quality in the area of Wikipedia see [31]. Rather, the present chapter is in the line of [18] who advise caution in using the Wikipedia category system in light of its Zipfian nature. We will consider this issue here as well. The chapter is organized as follows: Sects. 2 and 3 present our quantitative graph model. They include a series of newly invented measures of the topology of tree-like, DAG-like, and less restricted graph-like structures. Among others, this comprises measures of the multiplicity, the acyclicity, and the imbalance of graphs. Moreover, we also invent the notion of Zipfian bipartivity in order to explore the relation of social ontologies and the content units tagged by them. In Sect. 4, we utilize our quantitative graph model for the automatic structure-based classification of ontologies. This is done by analyzing formal, standardized, and terminological ontologies in relation to social ontologies. All in all, we classify 190 ontologies of 4 different types or classes of ontologies. Finally, Sect. 5 gives a conclusion and prospects for future work.

2 A Corpus of Social Ontologies on the Web In terms of graph theory, Wikipedia spans a highly typed graph which divides into two parts: the graph of content units (e.g. articles, portals, images) – henceforth called article graph – and the corresponding SOG or category graph. Categorization (as manifested by arcs between categories and categorized units) demonstrates that typing of vertices and arcs is important. One reason is that there may be links from article to category pages which actually do not indicate categorizations.2 Another reason is due to the variety of page types whose function departs from that of articles [49]. User, talk, and history pages, e.g., serve for the management of wikibased collaboration, whereas templates facilitate the production of the document network. Wikipedia contains many categorization links from categories to nonarticle units (e.g. to images or talks). However, article categorizations are in the majority. Any decision on taking some of these page types into account while disregarding

2

Such links are anchored within the body of an article, but not at the end of it where categorization links are located.

264

A. Mehler

Fig. 10.1 A schematic picture of the extraction scenario given by Wikipedia’s social ontology in relation to the corresponding article graph

others affects the topological analysis of the resulting graphs. Thus, this decision has to be wellfounded. In this section, we explain our method used to build a corpus of social ontologies and, thus, to justify the decision inherent in it. Look at Fig. 10.1 which gives a schematic account of a social ontology (i.e., top mode) in relation to the corresponding article graph (i.e., bottom mode of a bipartite graph). It has already been shown that wikis are characterized by a power-law-like distribution of the order (i.e. the number of vertices) of their connected components [49]. Below we will show that the very same holds for SOGs (see Sect. 3.2). This peculiarity challenges the significance of topological analyzes which are affected by the respective choice of the subgraph of the article or category graph. One reason to make a difference is that the numerous connected components of order 1 may be the result of noise, which interferes with attributing certain topological models to this graph. In order to control this source of noise, we distinguish the following variants of extracting category and article graphs: • Regarding the category graph, we can decide to extract it completely or to extract its Largest Weakly Connected Component (LWCC) only. In order to refer to the latter mode of extraction, we denote it by [c] while the former is denoted by [ ]. Table 10.1 contains a summarization of all extraction variants used here. Look at Fig. 10.1: by means of variant [ ] we extract the vertices u, v, w, x, y, z while x, y, z are extracted according to the variant [c]. • As far as we analyze SOGs in relation to their article graph – as done in Sect. 3.1 – we can apply the same extraction criteria to the article graph. By example of Fig. 10.1, this means to either extract all vertices 1, : : :, 7 or only the vertices 1, : : :, 4. The latter extraction variant is denoted by [a] (see Table 10.1).


265

Table 10.1 The different types of corpora derived from the corpus of social ontologies Mode Remark [ ] No special preprocessing [d] Article and discussion pages are interlinked by page-to-page links [r] All redirects are resolved [a] The largest weakly connected component of the directed article graph is extracted [c] The largest weakly connected component of the directed category graph is extracted

x 104 10 8 6 4 2 0 0

50

100

150

200

250

Fig. 10.2 263 SOGs sorted in ascending order according to the number of their vertices (their socalled order). The graphs have been extracted based on different extraction modes as distinguished in Table 10.1. The highly skewed distribution of their order demonstrates a power-law-like behavior as confirmed by a corresponding exponent 15:42. Thus, the order distribution of the SOGs is by no means Gaussian

• Both category and article pages may be interrelated by redirects which – unlike their nonredirecting counterparts – serve as the source of at most one arc. Thus, it may matter to resolve such redirect links (cf. [46]). For a path .x; r1 ; : : : ; rn ; y/ in which all rj ; j D 1::n, are redirects, to resolve them means to generate a single arc .x; y/ in the final graph. In order to distinguish this extraction variant and to make it combinable with other variants, we refer to it by the code [r]. • When taking all wiki pages into account (and not only some subgraph thereof) the question arises how to deal with discussion pages which are not explicitly linked to the page they discuss. In order to account for that, we consider the variant [d] that pages and their discussions are linked by two multiple arcs. Based on these preliminaries, we can distinguish the following extraction variants: [], [r], [dr], [ar], [cr], and [acr]. All in all we extracted 263 Wikipedias together with their social ontologies and derived for each of them 6 different SOGs. Figure 10.2 plots the distribution of their order.3 It shows that this

3

The order of a graph equals the number of its vertices [32].

266

A. Mehler

Table 10.2 The language codes of 160 Wikipedias whose social ontologies have been extracted. Underlined are those Wikipedias (except the English Wikipedia) whose SOGs are of an order larger than 100 if counted based on their [c]-variant. The language codes are taken from the Wikimedia project site. See http://meta.wikimedia.org/wiki/List of Wikipedias/sortable for the complete list of codes Language Code aa ab af ak als am an ang ar arc as ast av ay az ba bar bat-smg bcl be be-x-old bg bh bi bm bn bo bpy br bs bug bxr ca cbk-zam cdo ce ceb ch cho chr chy co cr crh cs csb cu cv cy da de diq dsb dv dz ee el eml en eo es et eu ext fa ff fi fiu-vro fj fo fr frp fur fy ga gan gd gl glk gn got gu gv ha hak haw he hi hif ho hr hsb ht hu hy hz ia id ie ig ii ik ilo io is it iu ja jbo jv ka kaa kab kg ki kj kk kl km kn ko kr ks ksh ku kv kw ky la lad lb lbe lg li lij lmo ln lo lt lv map-bms mdf mg mh mi mk ml mn mo mr ms mt mus my myv mzn na nah nap nds nds-nl ne new ng nl nn no nov nrm nv ny oc om or os pa pag pam pap pdc pi pih pl pms ps pt qu rm rmy rn ro roa-rup roa-tara ru rw sa sah sc scn sco sd se sg sh si simple sk sl sm sn so sq sr srn ss st stq su sv sw szl ta te tet tg th ti tk tl tn to tokipona tpi tr ts tt tum tw ty udm ug uk ur uz ve vec vi vls vo wa war wo wuu xal xh yi yo za zea zh zh-classical zh-min-nan zh-yue zu

distribution is far from being Gaussian. Rather, the order of SOGs is distributed in a skewed manner as captured by successfully fitting a power law with an exponent 15:42. One hundred sixty (160) of the SOGs which were extracted according to the mode [acr] contain at least 100 vertices. Only these SOGs have been considered to be of a reasonable minimal size to be analyzed structurally. The only exception is the English Wikipedia whose very large category graph makes it too complex to calculate all its cyclic structures by the algorithms used in this study. Therefore, it has not been considered here. This gives a Social Ontology Corpus (SOC) of 160 SOGs per extraction variant to be compared with the corresponding baseline graphs described in Sect. 4.1. Table 10.2 enumerates the language codes of the Wikipedia releases which have been considered. It also distinguishes those social ontologies which have not been considered.

3 Taking Topological Fingerprints of Social Ontologies We are now ready to introduce our quantitative model of SOGs, which will be used as a representation model of ontologies throughout this chapter. We do that to gain quantitative characteristics of social ontologies by means of which SOGs can be made an object of Quantitative Structure Analysis (QSA) [49, 52, 70, 72]. The idea is to enlarge the set of semiotic artefacts for which a purely structural analysis proves: (a) to be highly effective according to (b) a much smaller space complexity compared to content-based approaches based on the vector space model


267

Quantitative Structure Analysis (QSA)

Quantitative Generalised Tree Analysis Quantitative Tree Analysis (QTA) (QGTA) (Mehler et al. 2007; Pustylnikov & Mehler 2007)

Quantitative Network Analysis (QNA) (Mehler 2008b)

Quantitative Language Classification (QLC) (Pustylnikov & Mehler 2009)

Fig. 10.3 Quantitative structure analysis and its derivatives

or some of its derivatives. More specifically, we aim at instantiating quantitative structure analysis – see Fig. 10.3 – by graph models that, in complexity, lie between the simplicity of tree-like structures on the one hand and the complexity of unconstrained networks on the other. It turns out that this intermediate class of graphs relates to so-called generalized trees [23] and their extension in terms of generalized shortest path trees [50] (see Sect. 3.6). QSA is based on the idea of mapping graphs onto spaces of topological features by which they are made an object of the apparatus of supervised and unsupervised learning. Thus, a main step toward instantiating QSA by a novel graph class is to specify such topological features which guarantee separability between welldistinguished subclasses of the focal class of graphs. Basically, there are five reference points for exploring such features: 1. Bipartivity: A fundamental characteristic of SOGs is that they are built to categorize noncategorial units (texts, images, videos etc.). Thus, we start our quantitative analysis of SOGs with a focus on the kind of bipartivity spanned by the network of categories and categorized units. This is done in Sect. 3.1. 2. Complex network theory: Next, we study well-known characteristics of complex networks by example of SOGs. As SOGs depart from small worlds – they miss, e.g., the kind of transitivity which is constitutive for the latter [90], but not for SOGs – we do that nonexhaustively. This is done in Sect. 3.2 by examining the statistics of connected components. 3. Hypertext theory: As a reference to classical hypertext theory, we utilize two of its characteristics which have been introduced to study hierarchical systems [14]. As SOGs are hypertexts and resemble DAG-like structures, it is wellmotivated to test the separability of this model. This is done in Sect. 3.3. 4. Structure formation beyond trees and DAGs: A brief look on SOGs rapidly brings to light that they are cyclic.4 As any cyclicity stands in opposition to 4

Note that the documentation of the Wikipedia suggests the absence of cycles and that the category system of the Wikipedia spans DAGs (cf. http://en.wikipedia.org/wiki/Wikipedia:Categorization). We show more precisely the degree by which this is only approximately true.

268

A. Mehler

attributing SOGs to the class of trees, we put special emphasis on their degree of (a-)cyclicity. This is done in Sect. 3.4. 5. Finally, we combine information theory and complex networks by analogy to [22].5 That is, we define entropy-based measures in order to locate SOGs between perfectly balanced trees and imbalanced networks. This is done in Sect. 3.5.

3.1 Bipartition Generally speaking, a graph G D .V; E/ is bipartite if its vertex set V is partitioned into two nonempty disjunct subsets A; B such that every edge fv; wg 2 E ends at vertices v 2 A and w 2 B. Social ontologies naturally induce bipartite graphs in conjunction with the units categorized by them. That is we define G in such a way that A is the set of categories of a given SOG, whereas B is the set of categorized units, and E covers all categorization links whose orientation is abstracted. In this section, we shed light on the dynamics of the relation of categories and categorized units. We show that the topology of this bipartite graph is law-like in the sense that its subsets tend to be regularly connected irrespective of the SOG under consideration. That is, we hypothesize that SOGs span a sort of Zipfian bipartivity which makes them unique among the class of bipartite systems. In order to do that, we proceed as follows: We start with analyzing categories in relation to categorized units (Sect. 3.1.1). Then we reverse this process in order to look on categorized units in relation to categories (Sect. 3.1.2). Third, we analyze the contribution of categories to the cohesion of document networks and compare this with the role of conceptual units in terminological ontologies (Sect. 3.1.4). Finally, we bring to light the law-like kind of bipartivity spanned by SOGs (Sect. 3.1.3), which we call Zipfian. A short note on related approaches to bipartivity in complex networks: [34] proposes a measure of the degree of bipartivity in complex networks – see [27] and [26] for related approaches. This analytical approach departs from ours in the sense that we do not discover some unknown bipartivity in given networks. In contrast, we start from given bipartitions and ask for the type of bipartivity manifested by it.

5

This approach is in the line of [13] who develops information-theoretic indices of graphs and their topology. See also [87] for a related approach in the area of quantitative biology. A second impulse comes from [22] who calculates entropies of probability distributions of vertices in complex networks. Albeit this coincidence we deal with complex nearly DAG-like graphs apart from complex networks. In any event, it is our conviction that the analysis of graph structures can gain invaluable insights from these two approaches beyond of what has been done so far in complex network theory.


269

Our aim is to show that the sort of bipartivity of SOGs – which we call Zipfian – is law-like by analogy to the small world topology, which actually proves to be a law-like characteristic of social–semiotic networks [61].

3.1.1 Categories in Relation to Categorized Units You shall know a category by the company it keeps – in the case of SOGs, this means: by the articles it categorizes. This statement, a variation of a dictum of [29] on lexical items, assumes that categories are sufficiently separable by the articles typed by them. That is, whenever the community of wikilocutors invents a novel category there is a period of time after which this category is used sufficiently often so that it can be separated from other categories by its meaning. It turns out that this expectation fails in a certain sense when we look on SOGs. Figure 10.4a shows the results of fitting the model P .k/ k to the out degrees of category pages in relation to articles and other types of pages which belong to the same LCC. That is, fittings are computed for the [acr]-variant of the SOC. As indicated by the Adjusted Coefficient of Determination (ACD) (see the dashed line in Fig. 10.4), fitting is in all cases successful. Only in the case of smaller SOGs, do we get values for the ACD lower than :9. We also observe that in the majority of cases 1 < < 2. Only in the case of 4 SOGs is > 2. Note that ˛ < 2 indicates a lack of a finite expected value of the fitting theoretical distribution [62]. Note also that we perform power law fitting based on the complementary cumulative distribution [62]. Figure 10.4a demonstrates the Zipfian nature of categorization (cf. [88] for a related result). That is, the majority of categories is given by hapax legomena, i.e., by categories whose usage-based meaning (according to the dictum of Firth adapted to social ontologies) is hardly specifiable and discriminable as being used only once.

a

b

9 8 7 6 5 4 3 2 1 0

10 8 6 4 2 0 0

2

4

6

8

10 x 104

0

1

2

3

4

5

6

7

8 x 105

Fig. 10.4 (a), Solid line: Distribution of the exponents of the power laws fitted to the degree distribution of categories in relation to articles categorized by them. Dashed line: The corresponding values of the adjusted coefficient of determination. (b), Solid line: Distribution of the exponents of the power laws fitted to the degree distribution of articles in relation to categories by which they are categorized. Dashed line: as before. All fittings are computed for the [acr]-variant of the SOC

270

A. Mehler

The different SOGs do not differ with respect to this tendency, but with respect to its intensity. Regarding the predominant shape of the usage distributions of categories, we are in a perfect analogy to lexical systems, so we can speak of a topological invariant [86]. Thus, since categorization by social ontologies is a Zipfian process [59, 92], we see that the statement presented at the beginning of this section fails. The reason is that what is not specifiable by its use is hardly separable.

3.1.2 Categorized Units in Relation to Categories Analogously, in variation of Firth, we might state that you shall know a document (e.g., an article or portal) by the categories attached to it. Once more – by analogy to a finding of [88] – we see that this hypothesis is disputed by the findings reported in Fig. 10.4b. Because of the analogy to the findings about the category-to-article relation, we can only hint at the differences. It is obvious that the values of are now much larger – this indicates much more skewed distributions: either there is a larger number of articles which are categorized only once or – as is actually the case – the number of categories assigned to the most frequently categorized articles is not as high as the number of usages of the most frequently used category. In any event, the distribution of category assignments is highly skewed so that we can expect a large proportion of articles to be categorized only once. Thus, we get the following diagnosis: 1. Due to the findings reported in Fig. 10.9a, we conclude that by randomly selecting a category we probably retrieve one with the lowest out-degree. 2. Due to the findings reported in Fig. 10.9b, we conclude that by randomly selecting an article or another content unit from a Wikipedia we are likely to retrieve one which is categorized only once. So far, so good, as this finding is a well-expected result which coincides with a surge of related observations in complex network analysis. However, we may ask more specifically whether or not those rarely used categories are assigned to articles which, in turn, are infrequently categorized. The question of the kind of assortativity manifested by SOGs is studied subsequently.

3.1.3 Zipfian Bipartivity From Sects. 3.1.1 and 3.1.2 we know two conditions hold: C1 A SOG spans together with its article network a bipartite graph by means of categorization links. C2 In both directions, a Zipfian distribution is observed in terms of power lawfitting, that is, a highly skewed distribution with respect to the articles categorized by categories on the one hand and with respect to the categories used to categorize articles on the other.


271

From this point of view, its seems that we have yet another instance of a power law. However, things look different if we go one step further and ask whether the kind of bipartivity spanned by category and article graphs goes along with assortativity or disassortativity of vertex degrees. The latter notions have been formalized by [63] by means of a correlation coefficient. [63] applied them to nonbipartite complex networks. They show that social networks tend to disassortative degree mixing in the sense that like will be attracted to like. In this section, we apply the notion of assortative mixing to bipartite networks. That is, we introduce a notion of assortativity in bipartite graphs by relating it to the Zipfian nature of degree distributions as exemplified by SOGs. The basic idea is to call a bipartite graph Zipfian if it meets Condition C1 and C2 and if its vertices tend to mix either assortatively or disassortatively. In terms of SOGs assortative mixing means, for example, that high degree categories tend to be linked to high degree articles, while low degree categories tend to mix with low degree articles. This is exemplified in Fig. 10.5 by graph G1 in contrast to graph G2 in which the categorization links mix disassortatively (see below and the table embedded into Fig. 10.5). Obviously, there is some gradation in the formation of this sort of assortativity so that we make the following distinctions: • First-order Zipfian bipartivity: A bipartite graph is called Zipfian bipartite of the first order if the degrees of the vertices of both subsets of its partition are in each case distributed according to a power law. In order to simplify terminology, we distinguish the subsets of a bipartite graph G by calling the one its bottom and the other its top mode. The graphs G1 and G2 of Fig. 10.5 are first-order Zipfian bipartite.

Graph Order Size G1 G2 G3 G4 G5

30 30 30 30 30

r

58 0.13 70 −0.23 56 0.71 30 1 450 1

Fig. 10.5 Bipartite graphs G1 ; : : : ; G5 of the same order, but different size (i.e. the number of arcs) and their correlation values according to (10.1). The vertices 1–15 denote categories, those numbered 16–30 stand for categorized units

272

A. Mehler

• Second-order Zipfian bipartivity: A bipartite graph is called Zipfian bipartite of the second order if it is first-order Zipfian bipartite and if its vertices mix either assortatively or disassortatively by their degrees. In Fig. 10.5, for example, graph G1 is second-order Zipfian bipartite, but not graph G3 , as it is not firstorder Zipfian bipartite supposed that assortativity is measured by the correlation coefficient of [63] and that the out- as well as in-degrees are distributed according to a power law in G1 , but not in G3 .6 • Nearest Neighbor Zipfian bipartivity: A bipartite graph G is called nearest neighbor Zipfian bipartite if it is second-order Zipfian bipartite and if it meets at least the first of the following conditions: – Unipolar Nearest Neighbor Zipfian bipartivity: G is called unipolar nearest neighbor Zipfian bipartite if the average degree hknn i [68] of the nearest neighbors of vertices of degree k of either the top or the bottom mode are distributed according to a power law kNnn .k/ k either with a negative exponent (for 0) – so that mixing of vertices among the modes is disassortative – or a positive exponent (for 0/ – so that the mixing of vertices is assortative. – Bipolar Nearest Neighbor Zipfian bipartivity: The nearest neighbor Zipfian bipartivity of a graph G is bipolar if the latter sort of mixing occurs in both directions and not only in one. • Transitive Zipfian bipartivity: A bipartite graph G D .V; E/ is called transitive Zipfian bipartite, if it is second-level Zipfian bipartite and if there is a graph GO D O that meets at least the first two of the following additional conditions: .V; E/ – G is a subgraph of GO (note that both graphs have the same vertex set). – Unipolar transitive Zipfian bipartivity: G is called unipolar transitive Zipfian bipartite, if the average cluster value C.k/ of vertices of degree k [73] of vertices of only one mode linked to vertices of degree k of the other mode is distributed according to a power law C.k/ k with a negative exponent (for 0). It is this case which is of interest for the analysis of SOGs, by examining the cluster value C.k/ of articles as a function of the degree k of the categories to which they are assigned. In this case, infrequent categories which are presumptively of a semantic specificity to a high degree are mapped onto articles which are presumptively interlinked because of being semantically related. On the other hand, highly frequent categories diversify semantically. That is, they categorize many different articles which are unlikely linked because of this semantic diversification. – Bipolar transitive Zipfian bipartivity: The transitive Zipfian bipartivity of a graph G is bipolar, if the latter sort of clustering occurs in both directions and not only in one.

6

Note that the graphs in Fig. 10.5 denote different scenarios only schematically.


273

By means of the correlation coefficient of [63], we now formalize at the very least the notion of first- and second-order Zipfian bipartivity, leaving the formalization of the remaining notions to future work: Definition 2 (Zipfian Bipartivity). A graph G D .V; E/ is Z-bipartite if it is bipartite, that is, if there is a partition of V into nonempty disjunct subsets ?; > such that every edge fv; wg 2 E is ending at vertices v 2 ? and w 2 >, and if the following additional conditions hold: • Zipfian Bottom Mode: The probability that a vertex v 2 ? has the degree k is distributed according to a power law P? .k/ k ? . • Zipfian Top Mode: The probability that a vertex v 2 > has the degree k is distributed according to a power law P> .k/ k > . • Mode Mixing: The vertices of ? and > mix either assortatively or disassortatively in terms of their degrees. That is, either r.?; >/ 0 or r.?; >/ 0 where r.?; >/ D

1 m

P

1 m

P ei 2E

1 2 ei 2E 2 .ji

ji ki

1 P

1 i 2 .ji

m

C ki2 /

1 P m

C ki /

1 ei 2E 2 .ji

2

C ki /

2 2 Œ1; 1;

m D jEj and ei D fvj ; vk g denotes an edge ending at vertices vj 2 ? and vk 2 > of degree dG .vj / D ji and dG .vk / D ki . We call ? the bottom and > the top mode of G. Now, we are in a position to test hypotheses about the Zipfian bipartivity of SOGs and their corresponding article graphs. Because of the results reported in Sects. 3.1.1 and 3.1.2, it is obvious to say that SOGs tend to enter into first-order Zipfian bipartitions. Now, look at Fig. 10.6: it shows the distribution of the degree correlations of the 160 SOCs analyzed here. Obviously, Fig. 10.6 reports (if at all) a very slight tendency to disassortative mixing of vertex degrees – only some of the smaller SOGs of around 100–300 categories are second-order Zipfian bipartite as r takes values smaller than 0:4. However, things look different if we look at nearest neighbor Zipfian bipartivity. The results for fitting the corresponding models are reported in Fig. 10.7. While categories in relation to categorized units do not show the kind of disassortativity known from many complex networks [68] (see the results reported in Fig. 10.7a), the reverse case seems to be true regarding categorized units in relation to categories (cf. Fig. 10.7b). Now, we observe many SOGs with a -value greater than one and a corresponding adjusted coefficient of determination close to one. That is, in these cases, frequently categorized articles tend to be classified by categories which are infrequently used whereas infrequently categorized articles tend to be classified by those categories which are frequently used for this task. Thus, SOGs instantiate together with their corresponding document graphs the class of unipolar second-order (since disassortatively mixing) Zipfian bipartitions. Finally, have a look back on Fig. 10.6 which also reports the values of fitting the empirical distribution of C.k/. Obviously, this distribution is in a sense highly skewed in the case of some of the SOGs, so that we can additionally

274

A. Mehler

a

b

8

0

6

−0.2

4 −0.4

2

−0.6

0

−0.8

−2 −4

−1 0

2

4

6

8

10 x 104

0

2

4

6

8

10 x 104

Fig. 10.6 (a): The distribution of the degree correlations of 160 SOGs in the bipartite mode of the article and category graph – based on the [acdr]-variant. (b): Solid line: distribution of the exponents of the model C.k/ k fitted to the distribution of the average cluster value C.k/ of articles categorized by categories of degree k. Fitting is done for 160 SOG. Dashed line: the corresponding values of the adjusted coefficient of determination

a

b

2

6 5

1.5

4 1

3

0.5

2 1

0 −0.5

0 −1 0

2

4

6

8

10 x 104

0

2

4

6

8

10 x 104

Fig. 10.7 (a): Solid line: distribution of the exponents of the model kNnn .k/ k fitted to the distribution of kNnn .k/ values for 160 SOGs. Dashed line: the corresponding values of the adjusted coefficient of determination. (b): the same functions as in (a) but from the perspective of articles in relation to categories

speak of transitive Zipfian bipartitions. These are unipolar due to the fact that the reverse orientation misses this sort of transitivity – simply because of the fact that triangles are rather seldom in more tree-like SOGs. Note also that some of the SOGs demonstrate negative values of so that the exponent of the power laws under consideration becomes positive. What does that mean in terms of cognitive semantics? How can we interpret these results in the light of structure formation within semantic spaces as manifested by SOGs? A key notion to give an answer to these questions is that of semantic diversification [1]. Altmann [1] defines semantic diversification by analogy to [92] as a constantly, though slowly changing mapping between expression and meaning units. This mapping is seen to be characterized by a topological invariant in terms


275

Table 10.3 Four cases of mapping categories and texts Frequent categories Infrequent categories Frequent texts Assortative (Case A) Disassortative (Case B) Infrequent texts Disassortative (Case B) Assortative (Case A)

of a Zipfian distribution. It is easy to apply this notion to SOGs: Firstly, we conceptualize semantic diversification as a property of categories in relation to the variety of articles they categorize: if there are many such units, the category is said to diversify semantically – an extreme case is a category used to categorize every article. The reverse case is the one of a category which tends to semantic unification in the sense that the number of units categorized by it tends to approximate 1 – a situation which is very frequent in SOGs (cf. Sects. 3.1.1 and 3.1.2). Now, we distinguish the following prototypical scenarios: • Scenario A: SOGs which enter into assortatively nearest neighbor Zipfian bipartitions tend to show semantic diversification by their frequent categories. That is, categories which are frequently used to categorize different units are assigned to articles which are likewise frequently categorized by many categories. At the same time, infrequent categories are assigned to articles which are infrequently categorized (cf. Case A in Table 10.3). In a nutshell: in SOGs of the present scenario, semantic unification is demonstrated by infrequent categories in relation to rarely categorized units while semantic diversification affects their high frequent counterpart in relation to frequently categorized and, thus highly specified, units. In such SOGs, articles about some outlier topic of high semantic specificity are categorized by so to speak outlier categories of a likewise high semantic specificity, where this specificity is manifested by a low degree of usage. • Scenario B: In the reverse case of a disassortative nearest-neighbor Zipfian bipartite category-article graph, frequent, that is, semantically-diversifying categories are used to categorize infrequent articles while infrequent, that is, semanticallyunifying categories are assigned to frequent articles (that is, articles which are frequently categorized by categories which are semantically specified to a high degree) (cf. Case B in Table 10.3). Thus, the circumstances are reversed in this scenario of a SOG: semantic diversification relates to rarely-specified articles, whereas semantic unification affects the relation of categories to highly-specified articles. By looking at the SOGs analyzed here, we get evidence for scenario B, but not for A (see above). Why? Trying to give an answer to this question certainly deserves a separate article. It might help to hint at the notion of shareability of knowledge structures as introduced by [30]. Under the restriction of the Wiki medium and facing the effort to categorize articles, there may be the tendency to use more frequent categories of a lower level of semantic resolution, that is, categories near to the main category of the corresponding SOG. These categories are more easily accessed (starting from the main category), more likely to be

276

A. Mehler

encountered while processing given articles and, therefore, more easily shared among different wikilocutors. On the other hand, articles which are assigned many semantically-specific, but infrequent categories may justify this effort because their problematic content prohibits the latter kind of unspecific semantic categorization. However, in order to test these and related hypotheses, we need further empirical analyzes beyond the scope of the present chapter. These questions must be addressed separately in future projects.

3.1.4 Cohesion In this section, we examine what happens to the page graph if the category graph is deleted. From WordNet, for example, we know that the word graph looses its connectivity if all synsets are deleted (see below). In terminological ontologies of this sort, the conceptual units play a major role in providing cohesion, that is, in spanning the network. This is self-evident as WordNet focuses on semantic relations of words which are disconnected if these relations get lost. However, what role does a SOG play in this sense? That is, what impact does a social ontology have in terms of the connectivity of categorized units? We address this issue by examining the impact of deleting the category graph on the Largest Connected Component (LCC) of the remaining page graph. That is, for a graph G D .V; A/ as the representation of the article graph GA D .VA ; AA / and the category graph GC D .VC ; AC / of a language specific release of the Wikipedia such that V D VA [ VC and A D AA [ AC , we relate two proportions: firstly, lcc.GA / D jV 0 j=jVA j is the proportion of the number jV 0 j of vertices of the LCC of GA to the order of that graph. Secondly, lcc.GjA / D jV 00 j=jVA j is the proportion of the number jV 00 j of articles in the LCC of G connected by paths possibly crossing categories to the order of GA . Obviously, lcc.GA / lcc.GjA / so that we get a coefficient which computes the connectivity provided by the category graph GC to G as the proportion of the latter two coefficients: coh.GC ; G/ D 1

lcc.GA / jV 0 j D 1 00 2 Œ0; 1: lcc.GjA / jV j

(10.1)

The smaller the lcc.GA / in relation to lcc.GjA /, the greater the connectivity of the articles passed by categories, the higher coh.G; GC / and vice versa. We call coh.GC ; G/ the cohesion provided by GC to G. The results of applying this measure to the [dr]-variant of the SOC are reported in Fig. 10.8, which shows that for larger Wikipedias the size of the LCC is more or less stable. Only in the case of smaller Wikipedias is there an effect worth mentioning. In these cases, there is a tremendous effect on article cohesion by taking the category graph into account – somehow comparable to terminological ontologies. However, we get a mean of D :0476 together with a standard deviation of D :0999, which indicates small cohesion values. In other words, in the typical case of a release of a Wikipedia, the category graph provides only a tiny amount of cohesion to its article graph.

10 A Quantitative Graph Model of Social Ontologies Fig. 10.8 Cohesion values coh of the SOGs of 229 language specific releases of the Wikipedia with at least 100 articles

277

1 0.8 0.6 0.4 0.2 0 0

5

10

15 x 105

In the case of the two reference word nets analyzed here, we get a completely different picture: the degree of cohesion provided by the synset graph to WordNet is :998 1, whereas in the case of GermaNet, we get :9998 1. That is, by deleting the graph of synsets, we get rid of nearly all connectivity in the word graph. Thus, the index of cohesion defined by (10.1) sharply separates social from terminological ontologies.7 In summary, SOGs span a sort of add-on with little impact on the connectivity to an already highly connected graph. Other than in terminological ontologies, the cohesion of categorized units is mainly based on their immediate relations and only rarely based on relations mediated by conceptual units. This means that the article graph is a small world widely independent of the category graph. The former simply does not need the latter to generate its topological peculiarity. In other words: social ontologies do not matter for spanning small-world-like article graphs. The contrary is true for terminological ontologies. Thus, social ontologies play a completely different role from terminological ones. This finding is obvious to some degree. However, caution should be exercised as Wikipedia-based category graphs are used more and more frequently, either as alternatives to terminological ontologies or as complementary lexical resources to perform text mining tasks as, for example, lexical chaining [53]. The present study is in support of adaptations to be made to social ontologies in order to support this process properly.

3.1.5 Discussion Social ontologies span so-called Zipfian bipartite graphs together with the system of content units they tag. When we look at the relation of categorized and categorizing units, we observe a highly skewed bipartition in the sense that there is a subset of frequently-used categories, whereas the majority of them are used only once. At the same time, a small subset of content units is frequently categorized, whereas the majority of them are categorized only once. So if we pool all categories, it is

7

Note that we use the terms index and measure synonymously (cf. [41]).

278

A. Mehler

very probable that we randomly select a sort of “hapax legomenon” from the pool. In the reverse case, the same happens by pooling content units. Moreover, we have shown how these two distributions mix disassortatively and that the sort of semantic diversification manifested by social ontologies manifests two complementary types of tagging behavior, which enslave the dynamics of the Zipfian bipartition. In a nutshell: social ontologies transpose the Zipfian nature from the level of unipartite graphs – as known from many natural graphs [7] – to bipartite graphs. Beyond that, we have also studied the add-on feature of social ontologies which make them dispensable with respect to guaranteeing small world effects of the graph of content units. It may come to light that this is the beginning of a graph model with a comparable diagnostic significance as the small world model of complex networks – but now in relation to bipartite graphs. In any event, knowing the lawlike regularity of this behavior is of interest to all those approaches which try to utilize social ontologies in text and web mining – and the set of these approaches is rapidly growing.

3.2 Connectedness By analogy to [49] but with a focus on SOGs, we now apply the statistics of connected components. The idea is to ask whether SOGs behave as complex networks which are known for giant connected components [16]. At first glance, such a finding would be contra-intuitive as SOGs span conceptual hierarchies which are subsumed under a single main category. However, such a finding is motivated by the fact that unlike complex networks, SOGs basically manifest hypotactic relations. Thus, their size (i.e. number of arcs) should be smaller than in complex networks, say document networks, of the same order. This, in turn, makes disconnected components more likely. In order to tackle this question, we compute two indices. We start with considering the proportion lcc.G/ D jV 0 j=jV j of the number of vertices of the Largest Connected Component (LCC) LCC.G/ D .V 0 ; A0 / of G D .V; A/ to its order – it is well known that small worlds have large values of lcc [61]. We find the very same for SOGs as can be seen from Fig. 10.9a: first, we find that D :835 and D :243 and, second, there is a slight positive correlation ( D :265) of order and lcc so that we may say that the higher the order of a SOG the larger the proportion of the order of its giant component to the overall number of vertices. This indicates that disconnected components are more frequent than expected at first glance: the main category of a SOG is the entry point to a large number of its vertices but far from all. Note that we processed the unprocessed [ ]-variant of our SOC and abstracted from the orientation of arcs to compute these statistics. As a second index of connectedness, we compute the exponent S of the power law which best fits to the size distribution of the connected components of D. From the point of view of complex network analysis we can expect that the probability


a

279

b 7

1

6 0.8

5 4

0.6

3

0.4

2 0.2

1 0

0 0

2

4

6

8

10 x 104

0

2

4

6

8

10 x 104

Fig. 10.9 Part (a): The distribution of the fraction of the LCC in relation to the order of the [ ]-variants of 160 SOGs. The rightmost outlier at the bottom of figure (a) denotes the fraction of the LCC of the new (i.e., the Persian) language (cf. Table 10.2). Part (b): Dots on the straight line: The Distribution of the exponents of the power laws fitted to the distributions of the orders of the connected components of the 160 SOGs. Stars on the dashed line denote the values of the corresponding adjusted coefficient of determination. Note that 40 SOGs of the 180 SOGs in total have been removed as they consist of a single weakly connected component and, therefore, do not allow power law fitting

P .n/ of connected components of D of size n is distributed according to a power law [49]: P .n/ n :

(10.2)

This is widely confirmed by our SOC. Look at Fig. 10.9b: it shows the distribution of in conjunction with the corresponding adjusted coefficient of determination. In nearly all cases reported in Fig. 10.9b, the fitting is successful. Moreover, the values of are remarkably high, indicating the existence of a finite mean and variance of the order of LCCs [62]. However, there is also a subcorpus of 40 SOGs for which fitting is unsuccessful simply by the fact that these SOGs consist of a single LCC. In summary, we get two significant results by the statistics of connected components: 1. Due to the findings reported in Fig. 10.9b, we conclude that by randomly selecting a component from the set of connected components of a given SOG, we probably retrieve one of the lowest order. However, this is only true if we disregard the remarkably large number of 40 SOGs out of 160 which consist of a single connected component. 2. Due to the findings reported in Fig. 10.9a, we conclude that by randomly selecting a vertex of a SOG, we probably retrieve one of the giant component of that SOG. This finding is in direct support of utilizing social ontologies for building open topic models and related web mining tools. The reason is that by entering the SOG of a Wikipedia by one of its articles, one probably addresses its giant component.

280

A. Mehler

3.3 Compactness and Stratification In reminiscence of hypertext theory, we now apply the measures of compactness and of stratum as introduced by [14]. One reason to do this is that they were invented to separate network- and tree-like structures. Another reason is that compactness already demonstrated its potential in network classification [49]. So the question is what to expect from these measures in the area of classifying ontologies. We start with stratum st as a measure of the deviation of a digraph D D .V; A/ from an out-tree [6] with a single leaf (henceforth called linear tree) of equal order: st.D/ D 1 indicates zero deviation, whereas st.D/ D 0 indicates the absence of any stratification; in other words, all vertices equally serve as entry points to D. Formally, stratum is defined as follows: PjV j st.D/ D

i

jDi: D:i j 2 Œ0; 1 I lap.n/ D lap.n/

(

n3 4 n3 n 4

; n mod 2 D 0 ; else

(10.3)

where D is the distance matrix induced by D and lap.n/ is the linear absolute prestige [14] of a linear graph of equal order jV j D n – see [14] for the complete formalism of this measure. As disconnected components miss stratification in relation to each other by definition, we apply st to the largest connected component of a SOG, that is, to its [cr]-variant. The results are reported in Fig. 10.10a, which shows that stratum rapidly shrinks with the size of the SOG while generally taking very low values. This resembles a linear behavior which is not falsified by fitting a linear model – cf. Table 10.4, which also reports a negative but low correlation of size and stratum. This indicates that Stratum is not expressive enough to tell something about the topology of SOGs. However, the classification of ontologies as demonstrated in Sect. 4 will have a closer look on the expressiveness of st.

a

b

0.1

0.025

0.08

0.02 0.06

0.015 0.01

0.04

0.005

0.02

0 −0.005

0 0

2

4

6

8

10 x 104

0

2

4

6

8

10 x 104

Fig. 10.10 (a): distribution of the stratum values of the LCCs of 160 SOGs. (b): distribution of the corresponding compactness values. The correlation of both measures with the order of the SOGs is shown in Table 10.4 which also reports the adjusted coefficients of determination of fitting linear models to these distributions


281

Table 10.4 Correlations and results of linear fittings regarding six different models Compactness: Stratum: Proportion of vertices in cycles ([ ]-variant) Proportion of vertices in cycles ([c]-variant) Multiplicity index: Acyclicity index:

Model cp st pc .GŒ / pc .GŒc / mult acy

0.334 0.2 0.134 0.2 0.121 0.449

ACD :662 :974 :94 :961 0 :123

Related to the notion of cyclicity (see Sect. 3.4) is the notion of the compactness of a digraph D. It has been introduced by [14] as a measure of cross-referencing in a network. See [14] for a formal definition of compactness. As shown by [49], this measure has a high separability in network classification when applied to the LCC of a complex network only. In this chapter, we follow this approach by operating on the [cr]-variant of our SOC. The results of doing this are reported in Fig. 10.10b, which shows – as before in the case of stratum – that the compactness of a SOG takes very low values (although not as low as in the case of stratum so that the negative correlation with size is higher) and rapidly shrinks with its ascending order. Once more it seems that this second classic of hypertext theory fails to tell us something interesting about the topology of SOGs. However, whether this is really true will be examined in more detail in Sect. 4.

3.4 Acyclicity and the Multiplicity of Sources In Sects. 3.1 and 3.2, we analyzed SOGs in relation to categorized units. Then, in Sect. 3.3 we started to analyze the topology of SOGs separately. This was done by looking at their stratification as measured by st on the one hand and their compactness as measured by cp on the other. These measures tend to produce very low and evenly distributed values so that we might self-contradictorily imply that SOGs are both far away from being stratified and far away from being compact. Obviously, these classical measures of hypertext theory do not provide insights into the topology of SOGs; we have to look for alternatives. Therefore, we turn now to investigating this topology in more detail. We start with focusing on two related features of SOGs: • Based on the understanding that SOGs span conceptual hierarchies we expect that they should evolve as tree-like structures with, ideally, a single root and no cycles. So a first question is how SOGs depart from this ideal tree-like structure. This is done in Sect. 3.4.1. • Vertices with multiple direct predecessors, that is, vertices with an in-degree > 1, may indicate a strongly connected directed acyclic graph D D .V; A/ of an order larger than jV j1. However, it may also indicate the existence of cyclic structures

282

A. Mehler

which locate SOGs in the more general class of graphs. In Sect. 3.4.2 we address this question by means of a measure of acyclicity which also covers the impact of multiple direct predecessors and, thus, DAG-like structures.

3.4.1 How Tree-Like Social Ontology Graphs Are Directed trees have a unique source (i.e, a vertex with zero in-degree [32]) and no cycles. By a simple evaluation we see that SOGs have both multiple sources and cycles. Look at Fig. 10.14 which demonstrates a scenario with multiple sources by example of the LCC of the SOG of the Avar Wikipedia (coded by av) with 3 sources. In relation to the eccentricity [80] of the upmost source and the order of this component, these are many sources. Note that the wiki medium displays SOGs as tree-like structures. In order to systematize this observation we introduce the measure of multiplicity mult.D/ which for a weakly connected digraph D D .V; A/ is defined as follows: ( mult.D/ D

min.jS.D/j;maxx2S.D/ .x// maxx2S.D/ .x/

0

W jDj > 1 W jDj D 0

2 Œ0; 1

(10.4)

where S.D/ D fv 2 V j Àa 2 A W out.a/ D vg is the number of sources in D and .x/ the eccentricity of x. mult.D/ computes, so to speak, how many sources are used to span the graph D with the maximum “depth” maxx2S.D/ .x/. For trees T with root r and .r/ ! 1, mult.T / approximates 0. Conversely, if the number of sources exceeds the maximum of their eccentricities, mult is maximal (note that we suppose that D is weakly connected and does not contain loops). Such a SOG gets a gestalt as shown in Fig. 10.14. It demonstrates a SOG with a sort of “mirror effect” in the sense that it looks the same top-down and bottom-up so that the initially spreading paths finally converge. Figure 10.11 shows the distribution of the source

1 0.8 0.6 0.4 0.2

Fig. 10.11 Distribution of the source multiplicity index within the SOC of 160 SOGs

0 0

2

4

6

8

10 x 104

10 A Quantitative Graph Model of Social Ontologies Fig. 10.12 The proportion of the number of vertices in cycles to the order of the corresponding SOG. Stars denote the values of the [c]and dots the values of the [ ]-variant of the SOC

283

0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0

2

4

6

8

10 x 104

multiplicity index in our SOC based on exploring its [c]-variant.8 It demonstrates a remarkable variance of values in-between the extreme cases of 0 (indicating a treelike structure) and 1 (indicating a multiplicity of sources).9 We also note that there is a remarkable number of (44) SOGs with a source multiplicity of 1 – irrespective of the size of the graph (the corresponding size-multiplicity correlation is 0:12) – while only 2 SOGs have a multiplicity index of 0. Thus, having multiple sources is a frequent phenomenon in SOGs – which in this sense are far from being perfect tree-like structures. The second characteristic we look at are cycles which are likewise more frequent in SOGs than at first expected. Figure 10.15 shows the Japanese Wikipedia whose SOG contains a remarkably large cycle of more than 50 vertices. In order to grasp this aspect of cyclicity, we define an index which computes the proportion pc of vertices entering into cycles to the order of the corresponding SOG. The results of this index are displayed in Fig. 10.12. It shows that the proportion of vertices bound to cycles is little affected by looking at either the LCC by itself or the whole graph. It also shows that in the majority of SOGs the proportion of such vertices is rather small ( D :0096 ^ D :0213 in the case of the [c]-variant and D :0134 ^ D :0422 in the case of the [ ]-variant). Thus, cycles tend to be rare and affect only a small proportion of vertices. However, we have to say that this diagnosis too locates SOGs away from perfect tree-like structures. This result is somehow surprising for two reasons: firstly, it is supposed that category relations in SOGs manifest hypotactic relations of subordination. Secondly, the wiki medium used to manage social ontologies does not support managing cycles. Thus, we assume that cycles manifest a sort of noise due to the variety of users, their varying levels of expertise and due to their missing overview of the SOG to which they contribute.

8

Of course, every weakly connected component of the [ ]-variant of the SOC has at least one source. This information is already implicity explored by means of the connected component statistics. Thus, we focus on operating on the LCC of each SOG when calculating the multiplicity index. 9 Note that D :514 and D :3621.

284

A. Mehler

Because of these findings, we have to say that SOGs are not tree-like – however, they tend to be more tree-like than they are like unrestricted graphs. In order to measure this tendency more exactly we now focus on a measure which takes both into account: (source) multiplicity and cyclicity.

3.4.2 How DAG-Like Social Ontology Graphs Are Knowing the number of cycles in a SOG does not tell us their location. They may occur, for example, in the upper part of the SOG where they hinder the hierarchical traversal of its main fields of knowledge. Alternatively, they may occur in more specialized areas located at the “bottom” of the SOG so that they are more easily disregarded. Further, cyclic structures may characterize only a small subset of fields of knowledge while leaving the majority of them untouched. Conversely, they may be evenly distributed – irrespective of the field of knowledge under consideration. The same considerations apply to the multiplicity of sources: they may be concentrated in some section or dispersed across the whole SOG. This is exemplified in Fig. 10.13b in contrast to Fig. 10.13a: although both graphs have the same multiplicity index, they are quite different in terms of the subgraphs dominated by their sources. In this section, we address these alternatives by means of a single measure. This is done in terms of answering two related graph-theoretical questions: • How much information of a SOG is triggered by its different sources? • How many acyclic components does an SOG contain? As we know (from Sect. 3.4.1) that SOGs contain cycles, we may subsume this class of graphs under the notion of unconstraint digraphs without looking for any further constraints on structure formation within this class. Actually, this is a bad choice. The reason is that – as we have seen – SOGs do not have many cycles. Thus,

a

b

Fig. 10.13 (a): The SOG of the Avar (av) Wikipedia with 3 sources (filled circles) and mult D :6 (b): A graph with the same multiplicity index as graph (a)


285

Fig. 10.14 The LCC of the SOG of the Nepali (ne) release of the Wikipedia with 17 sources

Fig. 10.15 All cycles in the SOG of the Japanese Wikipedia of an order of at least two vertices

they are more like Directed Acyclic Graphs (DAG) than digraphs in general. That is, because of the existence of multiple sources, SOGs tend to be like DAGs.10 The similarity of a digraph to a DAG of the same order has been extensively studied under the notion of nearly acyclic directed graphs, and this is what we check now: to what degree are SOGs nearly acyclic. There are competing definitions of nearly acyclic digraphs in graph theory. Zelinka [91], e.g., calls a strongly connected digraph D D .V; A/ nearly acyclic at vertex u 2 V if u belongs to every cycle in D. In this case, cutting u makes D an acyclic graph D u.11 However, this

10

This also means that if we disregard the multiplicity of sources we may say that SOGs tend to be tree-like. 11 By D u we denote in the usual way the subdigraph of D induced by V n fug.

286

A. Mehler

notion does not tell anything about the number of cycles in D, which may be many more than can be observed in SOGs of the same order. An alternative definition of acyclicity based on Strongly Connected Components (SCC) is given by [85]. From our point of view, this notion is likewise inappropriate as SCCs are rarely seen in SOGs, which are much more tree-like than random graphs (cf. Sect. 3.5).12 Alternatively, we might consider more elegant graph-theoretical methods as, for example, computing the DAG-width of a graph as a measure of near acyclicity (cf. [10] and [65]). However, this approach is in the order of polynomials, thus, we need a more efficiently computable measure. The reason is that SOGs may contain hundreds of thousands of vertices. What we seek is a notion of near acyclicity which calls a digraph D nearly acyclic if the number cyc.D/ of its cycles is small in proportion to its order, that is, cyc.D/ jDj. Such measures are studied by [76] and [78]. In this chapter, we utilize [76] 1-dominator set-based measure, which is of the lowest complexity among the candidates considered by him. Computing the 1-dominator set of a digraph D D .V; A/ is in the order of O.m/, where m D jAj [76]. As we have to explore SOGs of hundreds of thousands of vertices, this choice is indispensable. Generally speaking, the 1-dominator set DŒ1 of a digraph D D .V; A/ is defined as DŒ1 D fDv j v 2 V ^ 8w 2 V W Dv 6D Dw ) Dv 6 Dw g

(10.5)

where for each v 2 V the subdigraph Dv of D is recursively computed as ; : : : ; Dv.i C1/ D Vv.i C1/ ; Av.i C1/ Dv.0/ D fvg; A.0/ v

(10.6)

where for IN.v/ D fw 2 V j 9a 2 A W in.a/ D w ^ out.a/ D vg, we set Vv.i C1/ D fw 2 V j IN.w/ \ Dv.i / 6D ; ^ IN.w/ Dv.0;:::;i / D [ikD0 Dv.k/ g(10.7) .i C1/

and where Av Finally, we set

.i C1/

A is the arc set of the subdigraph of D induced by Vv Dv

Dv.jDj/ :

.

(10.8)

Note that Dv is called acyclic in the sense that the subdigraph Dv v induced by .j / Vv n fvg is acyclic [76]. Note that there may be a 0 j < jDj such that Vv D ;. Each vertex v for which there is an acyclic structure Dv 2 DŒ1 is called a trigger vertex of D as it is the root of one of the maximal acyclic components of D, i.e., spanned by Dv . In other words, DŒ1 can be seen as a decomposition of D into acyclic components of D [76].

12 Note that [85] as well as [76] and [77] aim at improving shortest path algorithms by operating on nearly acyclic graphs but not on graph classification. Therefore, the deficiency we found is irrelevant for them.


287

Apparently, we are not far away from a measure acy.D/ of acyclicity of a digraph D, which might look as follows: ( acy.D/ D

1

jDŒ1 j1 jDj1

1

W jDj > 1 2 Œ0; 1: W jDj D 1

(10.9)

Obviously, acy.D/ D 1 if D is a tree while acy.D/ D 0 if D is a completely connected digraph. A problem with this approach is that an increase in the number of acyclic components Dv 2 DŒ1 is not only induced by cycles but also by multiple sources. Thus, we have to adopt another approach and define acy.D/ D jDj1

X

jDx j C jND .x/j C x2V

1

2 Œ0; 1

(10.10)

where NDC .x/ D fy 2 V j gd.x; y/ > 0g

(10.11)

is the set of all vertices y in D D .V; A/ connected with x. Generally speaking, (10.10) is a measure of acyclicity in the sense that it measures the quotient of the triggered vertices in relation to those that are connected. More specifically, for acy.D/ 0 we get the information that the vertices of D tend to trigger the vertices connected with them. In this case, D is both: nearly acyclic and nearly free of multiple sources. Conversely, by acy.D/ 1 we get the information that the vertices of D trigger only a small subset of the vertices connected with them. In this case, branching out from any such vertex rapidly enters into cycles or subgraphs of D triggered by other sources. Based on this approach, we get a measure of cyclicity and multiplicity of sources by defining: cyc.D/ D 1 acy.D/ 2 Œ0; 1

(10.12)

In order to exemplify this measure, look at Fig. 10.17 on Page 290: Vertex 1 of tree B triggers a single acyclic component which covers the whole tree. That is, in B the “main category” 1 triggers all information about that digraph so that BŒ1 D fBg. Things look different if we explore the digraph F D .V; A/ in Fig. 10.17, which resolves into 2 acyclic components, i.e., FŒ1 D f.f1; 2; 3; 4; 5; 6g; Af1;2;3;4;5;6g /; .f7; 8; 9; 10; 11; 12; 13g; Af7;8;9;10;11;12;13g/g where AV 0 is the set of arcs induced by V 0 V . Note that while B has a single trigger vertex 1, F has three thereof, namely 1, 7, and 13. Thus, vertex 1 is more informative (i.e., it triggers more information) in B than in F because of being uniquely defined. This is reflected in Table 10.5 which shows among other things that 1 D acy.B/ > acy.F / > acy.H / D 0 (H is completely connected). Now, look at graph G in Fig. 10.17 for which acy.G/ < acy.F /. This graph has two sources where the second source labeled by 4 has a remarkable effect because of being with vertex 3. Now, vertex 1 only

288

A. Mehler

Table 10.5 The acyclicity index and the imbalance indices of the 8 sample graphs of Fig. 10.17 (values are rounded to 3 decimal places). The imbalance measures are computed for the DAGlike representations of the input graphs. Note that in the case of graph G the imbalance measures are computed by starting from 1 as the main category without considering vertex 4 – from that perspective this graph is indeed perfectly balanced. The correction of imbalance measuring (see below) is computed for the DAG-variants only Balanced tree: Linear tree:

Completely: connected graph

Gi

acy

Id

Ia

Is

Io

Il

Iw

Iv

Ic

Idp

A B C D E F G H

1 1 1 .846 1 .686 .637 0

0 .21 1 .122 0 .122 0 0

0 .065 1 .047 0 .04 0 0

0 .369 1 1 0 1 0 0

0 .485 1 .225 .21 .225 0 0

0 .418 1 .203 .21 .188 0 0

0 .21 1 .122 .21 .122 0 0

0 .054 1 .04 0 .04 0 0

0 .21 1 .122 .21 .122 0 0

0 .735 1 .3 .275 .3 0 0

triggers vertex 2, so there are 5 cyclic components. Thus, the measure of acyclicity according to (10.10) also captures the impact of multiple sources. Finally, note that by jBŒ1 j < jFŒ1 j we know the number of acyclic components in these two graphs – thus, our approach actually tackles both questions posed at the beginning of this section.13 Applying Index (10.10) to the SOGs of our SOC, we get the distribution in Fig. 10.16. We see that social ontologies are actually nearly acyclic ( D :7975 and D :1153) – a diagnosis which has already been motivated by our short look on the topology of SOGs in the preceding subsection. However, index (10.10) confirms this diagnosis by means of a mathematically well-defined measure which locates SOGs by loc :7975 ˙ :1153 within the interval of perfect acyclicity (acy D 1) and perfect cyclicity (acy D 0). Thus, we also know that the tendency of SOGs to approximate the structure of DAGs of equal order is above indifference :5 loc as well as far from being perfect. This means that SOGs span a class of graphs which are constrained in a way that separates them less from the class of more constrained DAGs than from the class of more general digraphs. This also means that a certain amount of cyclicity is a natural characteristic of the way social ontologies structure their semantic universes. Unfortunately, present-day approaches to utilizing them (e.g. in text mining) are not equipped to handle this.

13 At this point one might ask why we do not use a simpler notion of cyclicity [38] by counting, for example, the number of edges to be deleted in order to make a graph acyclic? The simple reason is that index (10.10) is more informative about the DAG-like structure of a SOG as it additionally includes the impact of multiple sources.

10 A Quantitative Graph Model of Social Ontologies Fig. 10.16 Distribution of the acyclicity values within the SOC of 160 SOGs

289

1 0.8 0.6 0.4 0.2 0 0

2

4

6

8

10 x 104

3.5 Balance and Imbalance In this section, we examine SOGs in terms of their balance and imbalance respectively. Strictly speaking, a tree is balanced if all its leafs have the same geodesic distance to its root. We will call this distance the depth of a vertex. Such perfectly balanced trees are hardly observable empirically – among the ontologies analyzed here, only the DDC tree has it due to its artificiality. Thus, we need to take account of imbalance as a characteristic of the formation of DAG-like ontologies. In order to do that, we focus on two hypotheses: • Hypothesis I.1: We assume that the spanning of SOGs is law-like in the sense that the amount of their imbalance is – as a result of their evolution according to the principles of distributed cognition – independent of their order. • Hypothesis I.2: We also assume that this imbalance separates SOGs from nonsocial ontologies. In order to investigate Hypothesis I.1, we check how far the imbalance of SOGs is affected by their order (i.e. number of vertices). If for a given measure of imbalance (as defined below), both variables (order and imbalance) are uncorrelated, this may be due to a linear behavior (where the function values are ideally ordered parallel to the horizontal axis) or due to a point cloud. Thus, we get a test scenario for fitting a linear model to the imbalance values of SOGs as a function of their order. If this fitting fails, I.1 is falsified from the point of view of the respective measure of imbalance (in this case, imbalance varies by order in an uncorrelated manner). In order to investigate Hypothesis I.2, we have to analyze the degree by which SOGs are separated from the reference ontologies of Sect. 4.1. The idea is to check whether these different classes of ontologies can be separated when made an object of a classification which solely explores balance measures to get a feature representation of SOGs. This is done in Sect. 4. In order to implement this approach, we start from Fig. 10.17. It shows 8 graphs that coincide in their order but differ in their degree of imbalance: Tree A is a perfectly balanced tree in which each leaf has the same distance to the root. This also

290

a

A. Mehler

b

c

1

9

d

8

1

7 10

2

3

4

2

6

3

4

11

5

6

7

5

5

6

7

8

9

10

11

12

12

8

9

4

10 13

11

e

f

12

3

13

1

2

g

1

h

1

3

13

4 5

1 2

3

4

2

3

4

6 2

5

6

7

5

6

7

7

13 8

9

10

8

9

8

10

12 11

12

13

11

12

13

11

9 10

Fig. 10.17 Graphs of equal order varying in their acyclicity and degree of imbalance

holds for Tree C simply because it has a single leaf. Tree B is located in-between these extreme cases. It manifests, so to speak, a more skewed form of imbalance: Unlike its siblings, vertex 2 dominates a subtree. That is, a single branch originating in the root is deeply structured, whereas the remainder has a flat structure. It is natural to assume that SOGs are located – by analogy to Case B – in-between perfectly balanced trees on the one hand and linear trees (cf. Sect. 3.3) on the other. That is, SOGs are imbalanced in a yet unknown manner to a yet unknown degree. The reasons for this imbalance are certainly manyfold: on the one hand, the various fields of human knowledge differ according to their degree of elaboration and, thus, span imbalanced systems. On the other hand, Wikipedia is a dynamic system to which wikilocutors of varying degrees of expertise participate in such a way that at any point in time the different fields of knowledge are unevenly represented by active wikilocutors. Thus, we cannot expect that SOGs evolve in a balanced manner. There are certainly many other factors with an impact on the balancedness of SOGs, which interfere in a yet unknown way.14 In any event, if SOGs are characterized by a law-like degree of imbalance there must be a reference quantity by which this degree is measurable, that is, a quantity for which there exists a critical size at which the quantity no longer correlates with the order of the SOG (in the sense of Hypothesis I.1). In other words: whatever the exact nature of the imbalance of SOGs is, as far as it is law-like we expect that it nontrivially emerges as a relatively stable result of

14 A synergetic model [2,39] of this process could be a starting point to build an integrative model of all characteristics of structure formation considered here. However, at present this is out of reach.


291

distributed cognition – irrespective of the language community by which the SOG is generated and irrespective of its order. We now investigate which characteristics of SOGs have this property and which do not. In principle there are infinitely many candidate reference quantities of imbalance and for each quantity infinitely many measures to implement it. In the light of the discussion of Fig. 10.17 we focus on information–theoretical measures [22, 24]. More specifically, we utilize entropy as a measure of the imbalance of a tree or Directed Acyclic Graph (DAG) according to the following approach. Look at Tree A in comparison with Tree E in Fig. 10.17: although all their leafs have the same distance to the root, we can hardly call E balanced in the same manner as A because E has one strikingly wide subtree rooted by 3. Thus, we get width as a second reference quantity of imbalance beyond depth. Note that the reference point for this sort of imbalance is the distribution of the widths of the subtrees dominated by the “daughters” of the root. Obviously, entropy H is a measure which quantifies how balanced these values are: if the tree is balanced with respect to these daughter nodes’ values according to the operative quantity, we have a uniform distribution of values so that H goes to 1. Vice versa, if the tree is imbalanced with respect to the quantity, H goes to 0 and the respective distribution of values is skewed. This approach can be formalized as follows. Let D D .V; A/ be a directed graph and x 2 V a distinguished vertex of D (e.g. the root of D if it is a tree). Further, let Q W V ! Œ0; 1 be an interval-scaled function of vertices y 2 V onto the unit interval (as, e.g., the fraction of the order of the strongly connected component to which y belongs in relation to jV j) and ND1 .x/ D fxi1 ; : : : ; xin g D fxi j gd.x; xi / D 1g V

(10.13)

be the out-neighborhood [6] of neighbors of x in D whose feature values Q.xi / D qi , i 2 f1; : : : ; ng, are measured such that 8i 2 f0; : : : ; ng W qi 0 and n X

qi D 1

(10.14)

i D1

where gd.x; xi / is the geodesic distance of x and xi in D. Based on that, we get a feature vector q.x/ D .Q.xi1 /; : : : ; Q.xin //0 D .q1 ; : : : ; qn /0

(10.15)

as input to the relative entropy in order to measure the balance of x in D with respect to Q in the range of Œ0; 1:15 Pn qi log2 qi H.q.x// RH.q.x// D D i D1 2 Œ0; 1 (10.16) log2 n log2 n 15 At this point, we might also take the variance or mean of the feature vector q.x/ as an aggregation function to measure the balance of x. See [52, 70] for such an approach.

292

A. Mehler

Further, we naturally get a measure of imbalance IQ of x in D induced by Q. This is done by means of the redundancy measure R: IQ .x/ D R.q.x// D 1 RH.q.x// 2 Œ0; 1

(10.17)

By (10.17) we have a scheme for measuring the imbalance of a vertex x in a digraph D, which is instantiated by specifying the vertex-feature-generating function Q. That is, by varying instances of the variable Q we get alternative imbalance measures. In the following subsections we consider a set of such candidate instances. Amongst others, this relates to the depth, width, order, and dependency imbalance. The question will be answered as to which of these instances falsify Hypothesis I.1 and which do not. Further, Sect. 4 will answer the question as to which of these instances falsify Hypothesis I.2. In order to facilitate this test we define two sets in addition to NDC .x/ (cf. (10.11)), which depend on the choice of the target vertex x 2 V of the digraph D D .V; A/ to be processed. First, NDi .x/ D

fy j gd.x; y/ D i g ; i > 0 fxg ; else

(10.18)

is the set of all vertices y whose geodesic distance to x equals i . Second, C LC D .x/ D fy 2 ND .x/ j Àa 2 A W in.a/ D yg

(10.19)

is the set of all vertices in NDC .x/ (see (10.11)) whose out-degree is 0. Before applying this approach to SOGs, we have to preprocess them. The reason is that they are neither trees nor DAGs as has been shown in Sect. 3.4. Thus, we perform four preparatory steps of preprocessing SOGs in order to map them onto both a tree- and a DAG-like representation: 1. First, we account for the fact that SOGs are typically disconnected graphs (see Sect. 3.2). This is done by operating only on their giant component only. That is, we take the [c]-variant of the SOC as the input to measuring the imbalance of SOGs. 2. Second, as SOGs typically contain cycles (see Sect. 3.4.1), we identify a DAGlike skeleton dag.D/ for each SOG D by deleting all cycles as identified according to Sect. 3.4. This approach of additionally considering DAGs as inputs to the measurement of imbalance is motivated as follows: the DAG D in Fig. 10.17 is equal to the tree B except that in D the vertex 3 dominates the same vertices as vertex 2. That is, 2 and 3 are balanced to the same degree in D but not in B. As SOGs span such DAG-like structures it is worth considering effects of this kind (as will be done in the following sections). 3. Third, we have to account for the fact that the DAG-like skeleton dag.D/ of an SOG D may have several sources (see Sect. 3.4.2). In order to circumvent


293

this impediment and to get a tree-like representation tree.D/ of the LCC of an SOG D, we chose the main category r as a distinguished vertex and perform a breadth-first search starting from r in dag.D/.16 4. Fourth, as we focus on the daughter nodes of the main category r by computing the entropy of some of their characteristics, we eliminate linear tree-like prefixes starting from r in order to prevent always getting a maximum value of imbalance for this sort of graphs. Note that for the purpose of measuring the imbalance of the SOG D, we always take the perspective of this main category r – in the case of tree.D/ and dag.D/. Now we are in a position to consider imbalance measures in more detail. We start with such a measure in terms of the depth distribution of the daughter nodes of the main category. This serves as a blueprint for developing all related measures in this chapter.17

3.5.1 Depth Starting from depth as a reference quantity of measuring imbalance, we introduce the so-called depth imbalance measure for a distinguished vertex x 2 V (e.g. the main category in the case of a SOG) of a digraph D D .V; A/ by utilizing (10.16) as the operative measurement scheme.18 More specifically, we set Q d and define for any two vertices x 2 V and y 2 ND1 .x/: d.y/ D

1 C maxv2LC .y/ gd.y; v/ D P jND1 .x/j C z2N 1 .x/ maxv2LC .z/ gd.z; v/ D

D

D

1 C .y/ P 2 .0; 1: jND1 .x/j C z2N 1 .x/ .z/

(10.20)

D

Note that 1 is added in the numerator and jND1 .x/j in the denominator. The reason is to differentiate a tree T1 in which only one daughter v of the root has the eccentricity .v/ D 1, whereas the remainder daughters have zero eccentricity from a tree T2 in which .v/ 1 (all else being equal). Obviously, these trees are quite different in terms of their depth imbalance and this all the more the larger the value of .v/ in T2 . By using the entropy (see below) to derive a measure of depth imbalance, this difference would be neglected if we miss the latter correction.

16

Note that this main category is always unique. An obvious alternative to this approach would be to analyze the distribution of imbalance values in a SOG – this will be one reference point for future work. 18 Botafogo et al. [14] already utilized depth as a reference quantity of imbalance. However, they unnecessarily use a recursive function for defining it and miss demonstrating its empirical significance. 17

294

A. Mehler

a

b

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

1

1

0

0 0

2

4

6

8

c

10 x 104

2

4

6

8

10 x 104

0

2

4

6

8

10 x 104

d

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0

0 0

2

4

6

8

10 x 104

Fig. 10.18 The distribution of (a) the average depth Ia , (b) the complexity Ic , (c) the depth Id , and (d) the dependency imbalance Idp for 160 SOGs. Straight lines show linear fittings using bisquare weights for downweighting outliers (obviously the fitting is hardly successful – cf. Table 10.6)

Now, for ND1 .x/ D fyi1 ; : : : ; yin g we set d.x/ D .d.yk1 /; : : : ; d.ykn //0 and call Id .x/ D R.d.x//

(10.21)

the (degree of) depth imbalance of (x in) D. Figure 10.18c shows the degree of depth imbalance of the SOGs in the [c]variant of the reference SOC as a function of their order. It does so for the DAG-like representation of their LCC. Figure 10.18c also shows the best fitting linear model. Although this fitting is not successful, its Adjusted Coefficient of Determination (ACD) is much higher than in the case of the majority of the remainder imbalance indices (cf. Table 10.6). Additionally, Table 10.6 reports a nearly zero correlation of depth imbalance and order. So it is clear that the depth imbalance of a social ontology cannot be predicted by knowing its size. This is further confirmed by a nearly zero standard deviation of depth imbalance as shown in Table 10.6. That is, Hypothesis I.1 (see above) is not falsified by the DAG-like representation of the SOGs in our SOC – this also holds for their tree-like representations although this is not being reported here. Thus, until it is falsified by later tests, we retain the hypothesis that social ontologies are characterized by a low degree of depth


295

Table 10.6 All 9 measures of imbalance defined in this chapter, their correlation with the order of the 160 reference SOGs separately calculated for the tree-like and DAG-like representation of their [c]-variant. Further, the adjusted coefficient of determination (ACD) of fitting a linear model to the respective distribution of imbalance values as well as the mean and standard deviation of the respective imbalance values of the distribution of DAG-like representations Ind. (Tree) (DAG) ACD(DAG) (DAG) (DAG) Diagnosis Average depth imbalance: Ia 0.083 0.046 .475 .061 .056 C Complexity imbalance: Ic 0.059 .003 .634 .16 .135 .083 .08 C Depth imbalance: Id 0.091 0.047 .578 .439 .25 Dependency imbalance: Idp 0.071 0.206 .03 Length imbalance: Il 0.022 0.135 .06 .377 .232 0.095 0.113 .528 .057 .05 C Level imbalance: Iv .387 .237 Order imbalance: Io 0.003 0.145 .073 Shallow imbalance: Is 0.116 0.244 0 .354 .405 .042 0.078 0.03 .355 .231 Width imbalance: Iw

imbalance which tends to be independent of their size. SOGs tend to be balanced in terms of their depth. Their different branches tend to be specified to an equally maximal level: the different branches of these SOGs tend to be either equally deep or equally flat in structure. What does this finding mean in terms of web and knowledge mining? It means that, whatever the differences of the branches used to represent the different fields of knowledge in a SOG, they tend to be balanced in terms of their depth. In other words: any two randomly chosen daughter categories of the main category of a SOG, say economy and culture, coincide in terms of their depth – more or less irrespective of the order of the SOG. That is, whatever main field of knowledge is randomly selected from a SOG, we can expect that its depth is similar to that of its siblings. This means that there is a sort of characteristic level of maximal specification or depth of field within this SOG. This is striking as the different fields of knowledge are certainly built by different groups of wikilocutors of varying levels of expertise. Thus, we may hypothesize that this finding indicates a universal law of the depth by which topic hierarchies of a given community are specified, so to speak a community-related maximum level of specification. It is easy to see that this finding can be utilized by open topic models and related approaches of hierarchical classification. In the following sections, we will refer to findings of this sort as an instance of the Tendency to Topological Invariance (TTI) and we will look for other characteristics of imbalance in support of this tendency. As an alternative to Id , we first consider the average depth imbalance by using the mean instead of max as the operative aggregation function. This is done by setting Q a such that for any y 2 ND1 .x/: P 1 C C1 gd.y; v/ v2LC jLD .y/j D .y/ a.y/ D 2 Œ0; 1: P P jND1 .x/j C z2N 1 .x/ C1 gd.z; v/ v2LC .z/ D

jLD .z/j

D

(10.22)

296

A. Mehler

Based on that we get Ia .x/ D R.a.x//

(10.23)

as the (degree of) average depth imbalance of (x in) D. Other than Id .x/, Ia .x/ explores the average depths of the different fields of knowledge dominated by the main category. By Ia .x/ ! 0 Id .x/ we get the information that these fields of knowledge are balanced regarding their average and maximum level of specification. This is perfectly exemplified by Tree A in Fig. 10.17 in which the maximum depth equals its average depth. We might say that SOGs of this sort do not fray from the point of view of the depths of their leafs. Exactly this picture is confirmed by our SOC. In Fig. 10.18a, we see the distribution of the average depth imbalance in perfect analogy to the distribution of depth imbalance. This is confirmed by a nearly zero standard deviation in conjunction with a nearly zero correlation of average depth imbalance and order (see Table 10.6). Note that the ACD of fitting a linear model reported in Table 10.6 does not indicate a good fitting. However, it is among the highest ACDs observed here. Thus, we see once more that Hypothesis I.1 is not falsified, so we speak of a TTI (Tendency to Topological Invariance – see above) with respect to average depth imbalance. So far we have measured imbalance in terms of the distribution of maximal depths of certain subgraphs of the input graph D. Obviously, we may also look for the distribution of their minimal depths. This is done by using min instead of max as the operative aggregation function. That is, we set Q s and define for any two vertices x 2 V and y 2 ND1 .x/ D fyi1 ; : : : ; yin g: s.y/ D

1 C minv2LC .y/ gd.y; v/ D P 2 .0; 1 C z2N 1 .x/ minv2LC .z/ gd.z; v/

jND1 .x/j

D

(10.24)

D

such that Is .x/ D R.s.x//

(10.25)

is the (degree of) shallow imbalance of (x in) D where s.x/ D .s.yi1 /; : : : ; s.yin //0 is the input feature vector. Other than Id , Is explores the imbalance of the depths of those leafs of minimum specification. The corresponding results are reported in Fig. 10.19h and Table 10.6. The results do not merely indicate the worst ACD fitting of a linear model. In addition, there is a tendency to a negative correlation in conjunction with the highest value of the standard deviation observed among all candidates of imbalance indices considered here. Figure 10.19h shows this with remarkable clarity: with shrinking values of order the values of the index of shallow imbalance sway between 1 and 0. Thus, we conclude that Hypothesis I.1 is falsified by analyzing imbalance in terms of minimal depth. In summary, the average depth and the maximum depth imbalance, but not the shallow imbalance tend to be orderindependent characteristics of SOGs according to Hypothesis I.1.


e

f

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

1

297

1

0

0 0

2

4

6

8

10 x 104

g

1

0

2

4

6

8

10 x 104

0

2

4

6

8

10 x 104

h

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0 0

2

4

6

8

10 x 104

Fig. 10.19 The distribution of (e) the length Il , (f) the level Iv , (g) the order Id , and (h) the shallow imbalance Is for 160 SOGs. Straight lines shows linear fittings using bisquare weights for downweighting outliers (obviously the fitting is hardly successful – cf. Table 10.6)

3.5.2 Order Botafogo et al. [14] have already considered the Absolute Child Imbalance (ACI) as a measure of the imbalance of tree-like structures. For a vertex x, its ACI is recursively defined as a function of the child imbalances of its neighbors. It is easy to show that the child imbalance of a vertex x, as defined in this way, equals jNDC .x/j. Starting from this observation, we define a measure of the order imbalance of a vertex x by setting Q o where jNDC .y/j 2 Œ0; 1 C z2N 1 .x/ jND .z/j

o.y/ D P

(10.26)

D

so that we get Io .x/ D R.o.x//

(10.27)

as the (degree of) order imbalance of (x in) D where the vector o.x/ is defined in the usual way. Other than the indices considered so far, Io .x/ evaluates the imbalance of

298

A. Mehler

the structuring of the main fields of knowledge in terms of the number of categories in use. In this sense, a SOG is balanced (Io .x/ ! 0) if it uses approximately the same number of categories irrespective of the main field under consideration. By looking at Fig. 10.19g and Table 10.6 we see that this is not a characteristic of SOGs. Although correlation tends to be low, standard deviation is high so that this perspective on imbalance falsifies Hypothesis I.1.

3.5.3 Level and Width Now, we consider width and level imbalance. The idea is to call a SOG balanced in terms of width if the maximum number of vertices with equal distance to the root tend to be the same for all main fields of knowledge. If, additionally, the distances of these vertices tend to be alike irrespective of the main fields under consideration, we speak of level imbalance. More formally, we set Q w where w.y/ D

1 C maxi 2f0;:::;.y/g jNDi .y/j P 2 .0; 1; C z2N 1 .x/ maxi 2f0;:::;.y/g jNDi .z/j

jND1 .x/j

(10.28)

D

where .y/ is the eccentricity of y. Next, we define Iw .x/ D R.w.x//

(10.29)

as the (degree of) width imbalance of (x in) D where w.x/ is defined in the usual way. This allows for defining the level imbalance by setting Q v where v.y/ D

1 C arg maxi 2f0;:::;.y/g jNDi .y/j P 2 .0; 1 C z2N 1 .x/ arg maxi 2f0;:::;.z/g jNDi .z/j

jND1 .x/j

(10.30)

D

so that Iv .x/ D R.v.x//

(10.31)

is the (degree of) level imbalance of (x in) D where v.x/ is defined as usual. The results of applying these measures to our SOC are reported in Figs. 10.20, 10.19f and Table 10.6. These results are quite different: while we argue in the usual manner that Hypothesis I.1 is falsified when looking at imbalance in terms of width (cf. the corresponding value of ), the contrary is true if we look at level. There is a remarkable, nearly order-independent balance of the level of maximum order among all levels in a SOG. So if we randomly select a SOG we know by this observation that its main fields of knowledge will probably coincide in terms of this level of


299

i

Fig. 10.20 The distribution of the width imbalance Iw for 160 SOGs. Straight lines shows linear fittings using bisquare weights for downweighting outliers. (Note the similarity of width, length, and order)

1 0.8 0.6 0.4 0.2 0 0

2

4

6

8

10 x 104

highest specification – whatever fields of knowledge are considered. As a result, level joins (maximum) depth and average depth as a group of size-independent characteristics of social ontologies.19

3.5.4 Length and Complexity The next two measures are motivated by synergetic linguistics [40]. They have already been applied successfully in genre analysis [71] and text categorization [52]. In this chapter, we test this model for its separability in the area of SOGs. This includes the notion of [40] of length and complexity. Among others, [40] defines the length of a constituent v in a tree as the number of tokens (in our case leafs) within the scope of that node, whereas the complexity of v is defined as the number of its immediate constituents (in our case tails of arcs). Imbalance according to length means that the different main fields of knowledge fray from the point of view of the number of their leafs. In contrast, the main fields are balanced in their complexity if they tend to be equally structured into immediate subfields. Based on that we set Q l in the usual way where l.y/ D

1 C jLC .y/j 2 .0; 1 PD 1 jND .x/j C z2N 1 .x/ LC D .z/

(10.32)

D

so that Il .x/ D R.l.x//

19

(10.33)

Without following this line of research here, it may be interesting to consider this level of maximum order from the point of view of conceptual levels in prototype theory [74].

300

A. Mehler

is called the (degree of) length imbalance of (x in) D. As in the case of Id we apply a correction so that branches of varying length have a different impact if the remainder of daughters of the main category is of zero length. Next, we set Q c and define jND1 .y/j 2 .0; 1 1 z2N 1 .x/ jND .z/j

c.y/ D P

(10.34)

D

so that Ic .x/ D R.c.x//

(10.35)

is called the (degree of) complexity imbalance of (x in) D. By looking at Figs. 10.18b, 10.19e, and Table 10.6 we see that the complexity imbalance disperses to a much lower degree than the length imbalance. Further, whereas in the case of the latter we have a latent negative correlation, complexity does not correlate with size. Thus, we are disposed to falsify Hypothesis I.1 when instantiating imbalance by length, but not when doing so by complexity – there is at least a slight tendency that complexity can be conceived as a size-independent characteristic of the imbalance of social ontologies.

3.5.5 Dependency Terminological and social ontologies typically model relations of subordination among conceptual units. Wikipedia does this in terms of the subcategory relation, which subordinates categories to their immediate superordinate one. This hypotactic relation resembles the dependency of a satellite to its nucleus as investigated by dependency theory [55]. Thus, it is natural to apply measures of this area [3] to evaluate the imbalance of SOGs. This approach measures the degree of dependency dp.r/ of a directed tree T D .V; A; r/ rooted in r by evaluating two reference points: 1. The larger the value of jV j, the more vertices are subordinated directly or indirectly to r, the higher the value of dp.r/. 2. The more levels in T , i.e., the larger the maximum depth of its leafs, the higher the value of dp.r/. Obviously, an out-tree T of order n with a single leaf, i.e., a tree which spans a single simple path, is of maximum dependency among all trees of the same order. In this case, no vertices are subordinated to the same superordinate, that is, no vertices are co-dependent. Conversely, a tree of the same order and depth 2 is of minimum dependency. This observation makes dp an index of the topology of social ontologies in terms of the subordination of vertices – alike to the stratum measure but in a much simpler and, as we will see, more expressive manner.


301

In order to grasp the reference points [1,2] and [3], define a coefficient of relative dependency for the root x of a tree T D .V; A; x/ as O dp.x/ D

2

P.y/C1 ˇˇ i 1 ˇˇ i NT .x/ i D1 2 .0; 1 jV j.jV j C 1/

(10.36)

where jV j.jV2 jC1/ is the maximum absolute dependency degree a tree with jV j vertices can assume. We redefine this index in terms of information theory and DAGs as follows: we set Q dp and define for any two vertices x 2 V and y 2 ND1 .x/ D fyi1 ; : : : ; yin g: P.y/C1 ˇ i 1 ˇ i ˇNT .y/ˇ i D1 dp.y/ D P P.z/C1 ˇˇ i 1 ˇˇ 2 .0; 1: i NT .z/ z2N 1 .x/ i D1

(10.37)

Idp .x/ D R.dp.x//

(10.38)

T

Thus, we get

as the (degree of) dependency imbalance of (x in D), where dp.x/ is defined in the usual way. Obviously, the more the subgraphs dominated by the neighbors of x are alike in terms of their dependency structure, the higher the entropy of dp.x/, and the more balanced the graph. Vice versa, the less the entropy of dp.x/, the higher the dependency imbalance. The dependency imbalance of a DAG with a distinguished vertex x is affected, among others, by the widths, depths, and shallows of x’s neighbors. Thus, in an SOG with an imbalanced dependency structure the main fields of knowledge differ, among others, in terms of their width, maximal depth, and minimal depth: some fields are deeply organized including long paths starting from x while other fields are flat with only short terminological specification paths. In contrast to this, SOGs with a balanced dependency structure are those whose main fields of knowledge coincide, commonly, in being shallowly or deeply as well as broadly organized. This interpretation is supported by looking at Fig. 10.18d and Table 10.6. Because of the rather high value of and the slightly negative correlation of dependency and order, we conclude that Hypothesis I.1 is falsified if we instantiate imbalance by Index (10.38). That is, it does not seem that dependency as defined here is a size-independent characteristic of social ontologies, but rather it sways between both the extreme cases of complete balance and complete imbalance when looking for growing values of the order of SOGs. Notwithstanding this finding, we will see in Sect. 4 that both dependency measures (10.36) and (10.38) have a high potential in classifying ontologies.

302

A. Mehler

3.6 Synopsis In this section, we summarize our findings about social ontologies. As done throughout the paper, we focus on their graph structure: • Connectivity: By analog to small worlds in general [8], SOGs evolve around a single giant component which typically covers more than 90% of its categories. This also means that SOGs as based on hypotactic relations of conceptual subordination have many disconnected components (reflected by the Zipfian distribution of their order). Although components apart from the LCC of an SOG are relatively small, they are numerous and disconnected from its putative main category. This is certainly a result of the spontaneous organization of SOGs, which differs greatly from the controlled processes by which classification schemes evolve. • Multiplicity of sources: As a matter of fact each disconnected component has at least one source, and often several. In the same way, the giant component of an SOG tends to have multiple sources too. As a consequence, a single branch of an SOG may spring from different parents. Of course, polymorphism is a usual phenomenon in conceptual modeling. But it is not supported by the wiki media used by wikilocutors to manage SOGs as they work in a top-down fashion and, therefore, do not display multiple parents of a given concept. Rather, they show the numerous children of a given parent. Multiple sources are certainly likewise due to the spontaneous organization of SOGs as a result of distributed cognition among loosely interacting agents. Note that this finding is in conflict with approaches that utilize social ontologies for hierarchical classification by entering at some starting point to ascend toward the main category. Rather, this finding is in support of a preprocessing or special treatment of SOGs which reflects their proximity to DAGs. • Cyclicity: In the same line of diagnosing, we have stated that SOGs typically contain cycles, which means they depart in a strict sense from tree- and DAGlike structures. However, a short look at samples of social ontologies also reveals that these cycles are rare. Thus in a continuum spanned by DAG-like structures and unconstrained graphs, SOGs tend to appear closer to DAG-like structures. In order to make this a precise statement, we have invented an index of acyclicity which additionally grasps the impact of multiple sources. It shows that there is a tendency toward DAG-like structuring. It also shows that there is a remarkable amount of cyclicity and source multiplicity: top level nodes tend to trigger only 60–80% of the vertices mediately dominated by them. This raises an issue about the status of hyperlinks in social ontologies. Hyperlinks are untyped so they do not tell their actual status as hypertactic or paratactic or as any other kind of conceptual relation. That is, approaches to web mining uncritically treat each of these links as being of the same functional type – in fact this is not true. So instead of artificially building trees out of social ontologies by deleting multiple sources and cycles, this observation may be a starting point to type these links ex post in


303

terms of their semantics or function. Below, we make a proposal as to how this approach may help to find a more constrained class of graphs to formalize social ontologies. • Imbalance has been analyzed as a further reference point of structure formation in social ontologies. Apart from their (average) depth and level imbalance, the main fields of knowledge of SOGs tend to be imbalanced regarding their complexity, dependency, length, order, width, and minimal depth. As can be visually depicted by the sample in Fig. 10.14 on page 285, SOGs tend to grow by their width, but not by their depth, because the different fields of knowledge may be represented by subgraphs of different gestalt. That is, from the point of view of these main fields, social ontologies are unbalanced. This may hint at a sort of multidimensional Zipfian gestalt of graphs in this area as well.20 If this proves to be true one could hardly speak about well-developed branches of a social ontology. In terms of bad news this means that it is not possible to filter out loosely structured parts from a SOG as there is hardly something like a typical “branch of knowledge” or a typical structure of disciplines. • Bipartivity: The sort of bipartivity spanned by SOGs in relation to the units tagged by them is very characteristic. We have called this bipartition Zipfian because of the power-law-like n:m-relation of categories and categorized units. More specifically, we have shown that SOGs span together with the corresponding graph of content units the class of unipolar second-order, disassortatively mixing Zipfian bipartitions. This demonstrates the Zipfian nature of categorization. It also means that we must carefully decide how to enter the category graph via the article graph of Wikipedia. Although we have seen that cycles raise an issue about arc types, this disassortatively mixing Zipfian n:m-relation raises an issue about the role of the wikilocutors who produce the categorizations. The reason is that in order to explain the laws of this bipartition, we have to understand why wikilocutors use the categories they do. We must do this before we uncritically believe in each categorization as being checked by the community of wikilocutors or as being set, so to speak, by Laplace’s demon, who knows every category of every content unit. Following this line of reasoning means that we have to study the network of agents in close relation to the one of content units and categories as addressed by the notion of multi-level networks [49]. • Cohesion: A second aspect of the n:m-relation concerns the degree of cohesion provided by the category graph to the article graph. Under this regime, we have seen that social ontologies provide little cohesion to the article graph – SOGs are basically an add-on to an independently existing small-world-like network of content units [93], where the SOGs resemble DAGs without actually instantiating this class of graphs. In other words, the article graph does not need the SOG in order to be a small world.

20

See [54] who have introduced this notion in the area of modeling web genres.

304

A. Mehler

Fig. 10.21 Left side: a so-called generalized directed tree [50] with four up arcs, one down (d) and one lateral arc (la). Right side: a scenario which resembles the one of social ontologies, that is, a graph with two sources whose kernel (as a result of deleting all arcs typed as up or lateral arcs) spans a directly acyclic graph

Without formalizing SOGs in the light of this synopsis, we can finally ask more narrowly what class of graphs is instantiated by them. As has been shown, SOGs have multiple sources and cycles. However, they resemble tree- and DAGlike structures as their triggering vertices capture much information about their hierarchical structure. Thus it would be unnecessarily abstract to model social ontologies as unrestricted graphs. Having this in mind we may think about an extension of (directed) Generalized Trees (GT) as a proper graph-theoretical framework to capture the structural peculiarities of SOGs. GTs have been introduced as a class of graphs which combine the efficiency of tree-like structures with the expressiveness of general graphs [23, 50]. Basically, GTs build (a) on a tree-like skeleton by (b) adding up, down and lateral arcs. See Fig. 10.21 for an example of such a graph. An added value of GTs in the area of web mining is that their (undirected) edges and (directed) arcs, respectively, can be straightforwardly interpreted in terms of processes of information flow [50]. As a result, GTs are provided as a typed data structure beyond the more abstract and less informative graph models usually referred to in web mining [4]. This two-stage model of the generalization of trees can be naturally adapted to generalize DAGs as a model of social ontologies. This is exemplified in Fig. 10.21, which extends a generalized tree by four vertices among which is a further source labeled A. The idea is to conceive SOGs as connected GTs whose roots trigger information about different, but interlinked domains. As has been argued above, such graphs may be called Generalized Nearly Acyclic directed Graphs (GNAG)


305

as they augment a DAG-like skeleton by arcs which possibly introduce a small number of cycles. Following this line of thinking, it is possible to circumvent the pitfall of modeling social ontologies as trees by disregarding their full range of structure formation. These quantitative characteristics have been studied in the previous sections. However, this is future work.

4 You Shall Know an Ontology by the Topology it Spans In this section, we utilize the quantitative graph model of social ontologies in order to perform a quantitative structure analysis (see Sect. 3). More specifically, we show that social ontologies do indeed depart in terms of their topology from other wellknown ontologies which already existed before the advent of the web. That is, social ontologies are not only novel in terms of the process of distributed cognition by which they are generated but also in terms of the result of this process. In order to test this hypothesis, we perform two classification experiments by using the reference ontologies described in the following section. Next, Sect. 4.2 presents the classification experiment in detail.

4.1 Baseline Scenarios: The Usual Suspects In order to get a tertium comparationis for the classification of social ontologies, we analyze several formal and terminological ontologies. These reference or baseline ontologies are summarized in Table 10.7. Our idea is to perform a litmus test of the expressibility of our quantitative graph model by means of a classification which aims at separating social from these baseline ontologies. We do this by exclusively exploring quantitative characteristics of their topology. Therefore, we decide additionally to analyze ontologies which depart in terms of their semantics (i.e., by the object area they model) or pragmatics (i.e., by the process of distributed cognition through which they emerge). In the line of the basic assumption of quantitative structure analysis, we assume that these semantic or pragmatic differences correlate with the structural differences of the ontologies. In order to test this hypothesis, we analyze and compare the following classification schemes and ontologies: • The Dewey Decimal Classification (DDC) [66]: As a first reference ontology, we analyze the DDC which was invented as a classification scheme for subject cataloguing in libraries. It starts from ten main classes subdivided on two levels such that each subdivision is structured into ten areas. As a result, a forest of trees is spanned each of which has 111 vertices. In order to map this forest onto a single tree, we invent a virtual root which dominates all ten main classes of the DDC. Note that [81] subsumes DDC-like ontologies under the notion of classification schemes whose vertices denote conceptual types used to classify

306

A. Mehler

Table 10.7 The list of ontologies and their variants, respectively, which were used to provide a tertium comparationis for the classification of social ontologies. Order is the number of vertices and size the number of arcs in the corresponding graph used to represent the ontology. We also add the column type which classifies these ontologies in terms of the terminology of [81] about ontologies. In the second column we put the number of instances of the corresponding type in parentheses: it shows the number of ontologies classified in relation to the overall number of ontologies of the given type. The last row gives the number of social ontologies included in the classification experiment Ontology Type of Ontology Order Size O Dewey Decimal Classification Classification scheme (1/2) F DDC without virtual root M 1;110 1100 F DDC with virtual root M 1;111 1110 F Medical Subject Headings (MeSH) Classification scheme (1/2) F descriptors graph M 25;184 33;818 F tree number hierarchy (with virtual M 49;729 49;728 root) F OpenCyc Formal ontology (1/1) 54;914 76;136 O SUMO Formal ontology (21/21) F Communications (27) M 1;950 2;247 F Distributed Computing (124) M 2;047 2;343 F Economy (255) M 2;178 2;605 F Engineering Components (186) M 2;109 2;422 F Finance (204) M 2;127 2;435 F Geography (327) M 2;250 2;620 F Government (42) M 1;965 2;264 F Merge (the SUMO kernel) M 648 724 F MId-Level-Ontology (MILO) M 1;923 2;216 F Military (general) (31) M 1;954 2;251 F Military Devices (64) M 1;987 2;318 F Military Persons (16) M 1;939 2;233 F Military Processes (109) M 2;032 2;366 F North American Industrial M 1;927 2;223 Classification System (NAICS) (4) F People (401) M 2;324 2;616 F (Physical) Elements (109) M 2;032 2;326 F Transnational Issues (17) M 1;940 2;233 F Transportation (215) M 2;138 2;482 M 1;987 2;288 F Virus, Protein, and Cell Part (64) F WMD (157) M 2;080 2;405 F UNIfied ONtology (UNION) (2,328) M 4;251 5;004 F Wordnets Terminolog. ontology (6/6) F GermaNet 4.0 – nva synsets M 41;775 49;020 F GermaNet 4.0 – noun synsets M 27;824 34;706 F GermaNet 4.0 – verb synsets M 8;810 9;279 F GermaNet 4.0 – adjective synsets M 5;141 5;035 F WordNet 3.0 – noun synsets M 82;115 112;767 F WordNet 3.0 – verb synsets M 13;767 13;239 F Wikipedia Category Graph

Social ontology

(160/160)


307

topic areas. Using the DDC is of interest as its building principles are opposed to those of social ontologies – in terms of their time scale, their degree of control, formalization, and planning. Thus, we expect that this pragmatic difference is reflected by a topology-related classification. • Medical Subject Headings (MeSH) [60]:21 As a second instance of a classification scheme we analyze the MeSH, which was built to serve a DDC-like purpose for publications in the area of medical and biological sciences. Generally speaking, MeSH allows us to explore at least two kinds of structures. The reason is that the subject headings alias descriptors of the MeSH are mapped onto at least one vertex of the so-called Tree Numbers Hierarchy (TNH). This represents the kernel taxonomical structure of the MeSH – see [60] for more details. As a consequence, different subordination relations of the TNH ending at different tree numbers may end at the same descriptor. This gives a Directed Acyclic Graph (DAG) of the subject headings. Note that in the tree numbers hierarchy, we have added a virtual root to secure a single tree-like structure (otherwise we would have a forest). Table 10.7 reports the order and size of both variants of the MeSH analyzed here. • OpenCyc [67], i.e. the open source version of Cyc [44], models general knowledge as input to common sense reasoning. OpenCyc is the first instance of a formal ontology considered here. It has been selected in order to gain a further reference point to the classification of social ontologies. According to [81], a formal ontology is a kind of terminological ontology whose conceptual units and relations are axiomatically defined by means of a formal knowledge representation language. The reason to take formal ontologies into account is to go beyond a simple binary classification, that is, to have more than two classes or types of ontologies to work with. • The Suggested Upper Merged Ontology (SUMO)22 [64] is the second set of formal ontologies analyzed here. The SUMO defines a so-called upper or top-level ontology which covers concepts across special domains. In spite of this level of abstractness, this project includes several ontologies for special domains, which are related to the SUMO itself by the so-called MId-Level Ontology (MILO). In using the SUMO and these derivatives, we proceed as follows: any special ontology, say Transportation, is processed as an extension of the MILO, which in turn is processed as an extension of the SUMO kernel. However, we only take those special ontologies into account which extend the MILO and SUMO kernel, respectively. As a consequence, we did not process the World Airports A-K, World Airports L-Z, and the Countries and Regions which at the time of their download did not add any further subclass. Finally, we merged the SUMO kernel, the MILO, and all additional special ontologies into a single ontology – henceforth called UNIfied ONtology (UNION) – which covers all subclasses defined in any

21 22

Cf. www.nlm.nih.gov/mesh/. Cf. www.ontologyportal.org/.

308

A. Mehler

of these ontologies. As a result, we processed 21 ontologies of the SUMO project (cf. Table 10.7) based on the Standard Upper Ontology Knowledge Interchange Format (SUO-KIF). In Table 10.7, the number of subclasses which are specific to the corresponding derivative of the SUMO kernel and the MILO are bracketed. Note that we only processed the conceptual subclass hierarchy and its subordination relations. Thus, we left out all instances and otherwise typed vertices. As a result, the UNION contains 4,251 subclasses apart from the 20,000 terms managed by the SUMO project. • WordNet [28]: As a first instance of a proper terminological ontology we analyze the English WordNet. According to [81], ontologies of this kind are characterized by the absence of a fully axiomatic definition of their conceptual units and relations, which basically include the subtype–supertype and part–whole relation. One reason to distinguish these ontologies from their formal counterparts is that they model the lexical–semantic system of a language mindful of its redundancy, polysemy, and related features (which are better abstracted away in an upper ontology). Because of this semantic difference in addition to the pragmatic one mentioned above, we expect corresponding differences in the topology of formal, terminological, and social ontologies. A second reason to include WordNet is to analyze it as a bipartite graph of synsets in relation to lexical units. This is by analogy to social ontologies in relation to the content units categorized by their categories. Under this regime, we address the question of whether the bipartivity of social ontologies is outstanding and in which sense (see Sect. 3.1). A note on graph extraction by example of WordNet and GermaNet: As we basically analyze subclassification, subtype–supertype, or part–whole relations, we concentrated on these type of relations in WordNet and in GermaNet. Therefore, we did not extract, for example, antonymy, similar to, or attribute relations. A second reason to disregard relations of this kind is that they are paratactic (and per definition bidirectional23), but not hypotactic as the subtype–supertype or part– whole relations to be considered here. Supposed we take all relations among synsets into account, we would simply get a small world [83], that is, a complex network very much apart from any tree- or DAG-like structure – in this case, it would be a matter of simplicity to find out that they are disparate from the other types of ontologies studied here. Note further that in WordNet 3.0, the adjective-related synset graph does not contain hypernymy/hyponymy relations. So it has not been considered (the same holds for the adverb-related synset graph in WordNet 3.0 and in GermaNet 4.0 (see below)). • GermaNet [42]: Finally, we analyze GermaNet as a second instance of a terminological ontology. We do that by means of the GermaNet release 4.0 (see Table 10.7). Thus, we extracted the noun-, verb-, and adjective-related synset graph as well as the synset graph composed of these three subgraphs. All six terminological ontologies studied here are henceforth referred to as word nets [47].

23

Cf. http://wordnet.princeton.edu/man/wninput.5WN.html.


309

In summary, we distinguish four types of ontologies whose instances are made the object of an experiment in classification according to quantitative structure analysis: classification schemes (4 instances or variants, respectively), formal ontologies (22 instances), terminological ontologies (2 instances), and social ontologies (160 instances). A terminological note: we will henceforth speak of classification schemes as standardized ontologies in order to stress their position between formal and terminological ones. On the one hand, classification schemes are more formal than terminological ontologies as they avoid the kind of structuring which is typical for natural languages and, thus also grasped by WordNet and related projects. On the other hand, they miss the definitional strength of formal ontologies so that in this sense they resemble terminological ones. Finally, classification schemes have been invented in order to standardize certain information processes – that is why we call them standardized. Now, we are ready to perform a classification experiment in the line of quantitative structure analysis by the quantitative graph model presented in the preceding section.

4.2 Semisupervised Classification In this section, we present the results of an experiment in automatically classifying ontologies (Fig. 10.23). We do that in line with quantitative structure analysis (cf. [47]). This is basically done in four steps: S1 First, each of our target objects (in the present case, ontologies) is represented as a digraph. S2 Second, each of these digraphs is given as input to computing a subset of the topological characteristics described in Sect. 3. As a consequence, each target ontology is represented as a vector of the quantitative features of its graph structure. S3 Third, the feature vectors are made input to cluster analysis. This is semisupervised in the sense that the classification algorithm is informed about the number of target classes. This makes sense since the present approach does not explore new ontologies, but aims to identify the differences between the types of already known ontologies and the newly emerging type of social ontologies. In other words, we already know that formal, terminological, and standardized ontologies are different. What we do not know sufficiently is how they differ from social ontologies in purely structural terms. S4 Fourth, the F-measure value or F-score of the partition Cp , output by the last step, in relation to the target classification Ct is computed. See [36] for the details of computing this measure. The F-score is the harmonic mean of recall and precision; it is a well-established indicator of the quality of a classification. Note that we give equal weight to recall and precision.

310

A. Mehler

Index Feature F1 .D/ lcc.D/ F2

Area Def. OS(SOSH) OS(OSH) (1) jAj=jV j C D C (1) Sect. 3.2 D

F3 F4 F5

(3) (3) (2)

Sect. 3.3 C (10.3) (10.21) C

D C D D C

(2)

(10.27)

C

D C

(2)

(10.29)

D

(4)

(10.33)

D

(2)

(10.25)

C

D C

(4)

(10.35)

D

(2) (4) (2)

(10.10) (10.36) (10.38)

C C C

D C D C D C

(2)

(10.38)

C

D C

(2)

(10.21)

D

(2)

(10.27)

D

(2)

(10.29)

C

D C

(4)

(10.33)

C

D C

(2)

(10.24)

D

(4)

(10.35)

C

D C

(2)

(10.31)

D

(2)

(10.31)

C

6D

(2)

(10.23)

6D C

(2)

(10.23)

D

(2)

(10.4)

D

F6 F7 F8 F9 F10 F11 F12 F13 F14 F15 F16 F17 F18 F19 F20 F21 F22 F23 F24 F25

Short Description The average degree of D The fraction of the largest connected component of D cp.D/ The compactness of D st.D/ The stratum of D Idt .D/ The depth imbalance of D represented as a tree Iot .D/ The order imbalance of D represented as a tree Iwt .D/ The width imbalance of D represented as a tree Ilt .D/ The length imbalance of D represented as a tree Ist .D/ The shallow imbalance of D represented as a tree Ict .D/ The complexity imbalance of D represented as a tree acy.D/ The acyclicity index of D alt.D/ Altmann’s dependency index of D t Idp .D/ The dependency imbalance of D represented as a tree d Idp .D/ The dependency imbalance of D represented as a DAG Idd .D/ The depth imbalance of D represented as a DAG Iod .D/ The order imbalance of D represented as a DAG Iwd .D/ The width imbalance of D represented as a DAG Ild .D/ The length imbalance of D represented as a DAG Isd .D/ The shallow imbalance of D represented as a DAG Icd .D/ The complexity imbalance of D represented as a DAG Ivt .D/ The level imbalance of D represented as a tree Ivd .D/ The level imbalance of D represented as a DAG Iat .D/ The average depth imbalance of D represented as a tree Iad .D/ The average depth imbalance of D represented as a DAG mult.D/ The source multiplicity index of D

Fig. 10.22 The list of quantitative features of ontologies considered in the present study. They fall into three groups, that is, features of complex network theory (1), quantitative structure analysis (2), hypertext structure analysis (3), or features of quantitative linguistics (4) as indicated in the 4th column. Column 6 and 8 indicate whether the feature belongs (C) to the best performing subset of features or not () with respect to testing the SOSH and the OSH, respectively


311

Fig. 10.23 Schematic depiction of quantitative structure analysis: different networks are mapped onto points in an n-dimensional feature space which allows for computing the structural dissimilarities of these networks. In the present study, we chose a 25-dimensional feature space spanned by the features enumerated in Fig. 10.22

As we have already described this algorithm in detail in [49], we concentrate now on the peculiarities of its adaptation to classifying ontologies. This is basically done by formulating two target hypotheses to be tested. In the area of ontologies they read as follows: • Social Ontology Separability Hypothesis (SOSH): Social ontologies can be reliably separated from the set of formal, standardized, and terminological ontologies by means of the topological characteristics of the graphs spanned by their conceptual units. • Ontology Separability Hypothesis (OSH): Ontologies of different types, that is, formal, standardized, terminological, and social ontologies can be reliably separated by means of topological characteristics of the graphs spanned by their conceptual units. We view any of these hypotheses being falsified if the following inequality does not hold for the F-score F .Cr / of the corresponding random (“baseline”) classification Cr in relation to the F-score F .Cp / of the computed partition Cp : 0 < F .Cr / :90 F .Cp / F .Ct / D 1:

(10.39)

312

A. Mehler

F .Cr / is obtained by averaging the F-score over 1,000 repetitions of randomly distributing the input objects over the given number of target classes. Note that we compute this random distribution by informing the random distributor about the cardinality of the target classes. In the case of the SOSH it randomly distributes over two, in the case of the OSH it distributes over four target types or classes which altogether contain 190 objects (i.e. target ontologies) – cf. Table 10.7. Thus, the random distributor knows, for example, that there are 160 social ontologies which it needs to select randomly – of course, this information is not given to the classifier which computes Cp . Following this approach gives higher F-score values of this baseline classification than would be obtained without it.24 Why do we use this kind of baseline? The reason is that – to the best of our knowledge – we are the first who perform such a classification of ontologies. Therefore, we cannot refer to any related result in the literature. Inequality (10.39) means that if the F-score of the computed partition Cp is not equal or larger than 90%, as well as above the corresponding random partition, then we view the focal separability hypothesis as being falsified subject to the representation model of ontologies and the classification algorithm in use. In this case, our model either performs worse than a random classification of the sort done here, or it fails to classify a minimum of 90% of the objects correctly. Note that F-scores of at least 90% are generally known as being high-degree values. So it is reasonable to expect at least such a classification result. Note further that we demand that F .Cr / F .Ct /. That is, a good classification result should not be obtained by just randomly assigning objects to the target classes. However, we have to realize that F .Cr / is simply raised by highly disparate numbers of objects in the target classes (see below). Note that by falsifying the SOSH we do not necessarily falsify the OSH, although the latter is the stronger hypothesis (Table 10.11). The reason is that, because formal, standardized, and terminological ontologies are highly disparate, separating them from social ontologies as homogeneous members of the same class could be difficult. In order to test the SOSH and OSH subsequently, we represent each ontology by the 25 features summarized in Fig. 10.22. More specifically, we built four target classes (cf. Table 10.7): (a) two objects in the class of standardized ontologies (the DDC with virtual root plus the tree number hierarchy of the MeSH), (b) 22 objects in the class of formal ontologies (that is, the OpenCyc together with the 21 derivatives of the SUMO, (c) Six objects in the class of word nets and, finally, (d) the 160 SOGs analyzed above in the class of social ontologies. This gives an overall set of 190 objects to be classified. We start with testing the SOSH. The results of the corresponding experiment are summarized in Table 10.8. Obviously, there is a best performing approach with an F-score higher than 90% which also outperforms the corresponding baseline classification, which has an F-score of 0.75111. The reason that the latter F-score

24 Thus, the reader should not confuse Cr with a random uniform distribution of the objects over the target classes.


313

Table 10.8 Testing the SOSH by example of the class of terminological, standardized, and formal ontologies in relation to social ontologies so that we get two target classes of 190 objects (second column). Each ontology is represented by the 25 features of Fig. 10.22. The first column denotes the F-score of the classification. The third column denotes the clustering method while the fourth column shows the method of measuring object distances used by it F-Score .92648 .92648 .81395 .75111

#Classes 2 2 2 2

Clustering Method Hierarchical (Ward) Hierarchical (Ward) k-means (Cityblock) Cr

Object Distance Hamming distance Jaccard coefficient-based Mahalanobis distance –

Table 10.9 F-score of the optimal feature subset regarding the SOSH F-Score #Classes Clustering Method Object Distance .93391 2 Hierarchical (Ward) Jaccard coefficient-based Table 10.10 Testing the OSH by example of four classes (cf. second column), that is, social, terminological, standardized, and formal ontologies. These were represented by means of the quantitative graph model of Sect. 3. The remainder of columns have the same meaning as in Table 10.8 F-Score .80023 .80023 .80023 .73729

#Classes 4 4 4 4

Clustering Method Hierarchical (centroid) Hierarchical (median) Hierarchical (centroid) Cr

Object Distance Chebychev distance Mahalanobis distance Mahalanobis distance –

Table 10.11 F-score of the optimal feature subset regarding the OSH F-Score #Classes Clustering Method Object Distance .93346 4 Hierarchical (weighted) Jaccard coefficient-based

is that high is twofold: First, there are many more SOGs than ontologies of other types which makes random mappings more likely to be correct. Second, the random classification is informed about the exact cardinalities of the classes in the target classification Ct (see above). Therefore, it is comparatively easy to randomly select a SOG of a corpus of 190 ontologies in which 160 belong to this target class. In any event, as we meet Condition (10.39) we do not reject the SOSH but retain it until it is falsified elsewhere.25 Next, we consider the OSH. The corresponding results are summarized in Table 10.10. We see that the OSH is falsified since the F-score of the best performing classification model is less than 90%. However, we remember that we 25

In Table 10.9 we present the F-score of the best performing subset of features. This subset is reported in the 6th column of Fig. 10.22. We see that, by representing the target ontologies by this feature subset, the F-score is raised. Below we discuss this result in the context of testing the OSH.

314

A. Mehler

have used all features of Fig. 10.22 to represent and classify the objects. This is contradicted by the findings of Sect. 3.5, which have shown that several imbalance measures performed badly as order-independent characteristics of SOGs. Therefore we decided to perform an evolutionary search for the best performing subset of features in Fig. 10.22. The results of this F-score-related optimization are reported in the last column of Fig. 10.22. It displays a subset of 13 features. This gives us a much higher F-score of 0.93346 for the same classification experiment. Looking at the OSH from the perspective of this result, we do not reject it. It is worth looking at this better-performing subset of features (as displayed in the last column of Fig. 10.22) in more detail: surprisingly, compactness is among these features. We also find several imbalance indices which operate on tree-like representations of the input ontologies. However, in correspondence with the findings of Sect. 3.5 we see that the average depth and the maximum depth (though not the level imbalance) are among these best performing features complemented, among others, by the dependency imbalance and, more surprisingly, by the order and shallow imbalance. The fact that dependency measuring is so informative is additionally supported by the good separability provided by Altmann’s dependency index (see (10.36)) and by the fact that this role is confirmed by the tree- and DAG-like representations of the ontologies. Further, we see that the acyclicity index is also among the best performing features. However, we also get the result that the multiplicity is not informative in this sense. In a nutshell: some imbalance- and all dependency-related indices as well as the acyclicity index are among the most informative indices and are, therefore, worth studying in more detail. This evaluation is confirmed by the best performing subset of features regarding the SOSH (see the sixth column in Fig. 10.22): except from feature F22 and F23 the respective subsets coincide (see the seventh column in Fig. 10.22). Note that as reported in Fig. 10.22, we did not use quantitative features of the undirected representations of the digraphs being considered. This relates, for example, to the connected component statistics and here especially to the exponent of the distribution of the size of connected components. Taking these features into account may even better our classification. However, since the F-score is greater than 0.90 we are back to the position that neither the SOSH nor the OSH have been falsified. So we can neither reject the SOSH nor the OSH. Thus, we retain the hypothesis that social ontologies are a novel genre on the web which in more than 90% of the cases studied here can be reliably separated from formal, standardized, and terminological ontologies (which existed before the rise of the web) – in purely structural terms.

5 Conclusion This chapter has presented a quantitative graph model of Wikipedia-based social ontologies as a novel web genre which extends the set of formal, standardized, and terminological ontologies. This graph model focuses on the relation of categories


315

to categorized units as well as on the topological characteristics of the category graph itself. We have invented a series of novel measures of the topology of treelike, DAG-like, and less restricted graph-like structures. Among others, this includes measures of the multiplicity, the acyclicity and the imbalance of graphs. Further, we have shown that social ontologies are characterized by a sort of bipartivity which we have called Zipfian and that this bipartivity goes along with disassortative mixing of categories and categorized units – at least from the point of view of categorized units. Further, we have studied the impact of cycles and multiple sources in social ontologies as well as their special sort of imbalance. One of the recurrent themes of discussing the effects of these characteristics has been about web content mining, which faces almost insolvable problems when using the Wikipedia category graph for this task. There is a further implication of the findings presented here. This relates to approaches to language evolution which may aim at simulating the generation of social ontologies in completely artificial settings. Such simulation models need quality factors by which the performance of alternative simulation models can be evaluated. Further, they need knowledge about constraints of structure formation in real ontologies. The present chapter has also provided results in this line of research – beyond the sufficiently known statistics of power laws. Acknowledgment Financial support of the German Federal Ministry of Education (BMBF) through the research project Linguistic Networks, of the German Research Foundation (DFG) through the Excellence Cluster 277 Cognitive Interaction Technology (via the Project KnowCIT) and of the SFB 673 Alignment in Communication (via the Project A3 Dialogue Games and Group Dynamics and X1 Multimodal Alignment Corpora: Statistical Modeling and Information Management) is gratefully acknowledged. We also thank Dietmar Esch, Tobias Feith, and Roman Pustylnikov for the download of ontologies as well as Rüdiger Gleim, Olga Abramov, and Paul Warner for their fruitful hints which helped to reduce the number of errors in this chapter.

References 1. Altmann, G.: Semantische Diversifikation. Folia Ling. 19, 177–200 (1985) 2. Altmann, G., Köhler, R.: “Language forces” and synergetic modelling of language phenomena. In: Glottometrika, vil. 15, pp. 62–76. Brockmeyer, Bochum (1996) 3. Altmann, G., Lehfeldt. W.: Allgemeine Sprachtypologie. Fink, München (1973) 4. Baldi, P., Frasconi, P., Smyth, P.: Modeling the Internet and the Web. Wiley, Chichester (2003) 5. Bales, M.E., Lussier, Y.A., Johnson, S.B.: Topological analysis of large-scale biomedical terminology structures. J. Am. Med. Informat. Assoc. 14(6), 788–797 (2007) 6. Bang-Jensen, J., Gutin, G.: Digraphs. Theory, Algorithms and Applications. Springer, London/Berlin (2006) 7. Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999) 8. Barrat, A., Barthélemy, M., Vespignani, A.: Dynamical Processes on Complex Networks. Cambridge University Press, Cambridge (2008) 9. Barthélemy, M.: Betweenness centrality in large complex networks. Eur. Phys. J. B 38, 163–168 (2004)

316

A. Mehler

10. Berwanger, D., Dawar, A., Hunter, P., Kreutzer, S.: DAG-width and parity games. In: Durand, B., Thomas, W. (eds.) STACS, vol. 3884, Lecture Notes in Computer Science, pp. 524–536. Springer, Berlin (2006) 11. Bickhard, M.H.: Social ontology as convention. Topoi 27(1-2), 139–149 (2008) 12. Blohm, S., Kroetzsch, M., Cimiano, P.: Integrating the fast and the numerous – combining machine and community intelligence for semantic annotation and Wikipedia: Folksonomy meets rigorously defined common-sense. In: Proceedings of AAAI 2008 Workshop on Wikipedia and Artificial Intelligence: An Evolving Synergy (WikiAI08), Chicago (2008) 13. Bonchev, D.: Information Theoretic Indices for Characterization of Chemical Structures. Research Studies Press, Chichester (1983) 14. Botafogo, R.A., Rivlin, E., Shneiderman, B.: Structural analysis of hypertexts: Identifying hierarchies and useful metrics. ACM Trans. Inform. Syst. 10(2), 142–180 (1992) 15. Budanitsky, A., Hirst, G.: Evaluating WordNet-based measures of lexical semantic relatedness. Comput. Ling. 32(1), 13–47 (2006) 16. Caldarelli, G.: Scale-Free Networks. Complex webs in nature and technology. Oxford University Press, Oxford (2008) 17. Capocci, A., Caldarelli, G.: Folksonomies and clustering in the collaborative system CiteULike. J. Phys. A Math. Theor. 41, 224016 (2008) 18. Capocci, A., Rao, F., Caldarelli, G.: Taxonomy and clustering in collaborative systems: the case of the on-line encyclopedia Wikipedia. Europhys. Lett. 81, 28006 (2008) 19. Cattuto, C., Barrat, A., Baldassarri, A., Schehr, G., Loreto, V.: Collective dynamics of social annotation. PNAS 106(26), 10511–10515 (2009) 20. Cattuto, C., Benz, D., Hotho, A., Stumme, G.: Semantic grounding of tag relatedness in social bookmarking systems. In: The Semantic Web – ISWC 2008, vol. 5318, Lecture Notes in Computer Science, pp. 615–631. Springer, Berlin, Heidelberg (2008) 21. Chernov, S., Iofciu, T., Nejdl, W., Zhou, X.: Extracting semantic relationships between Wikipedia categories. In: 1st International Workshop: From Wiki to Semantics (Sem Wiki 2006), co-located with ESWC 2006, Budva, Montenegro, June 12, 2006 22. Dehmer, M.: Information processing in complex networks: Graph entropy and information functionals. Appl. Math. Comput. 201, 82–94 (2008) 23. Dehmer, M., Mehler, A.: A new method of measuring the similarity for a special class of directed graphs. Tatra Mountains Math. Publ. 36, 39–59 (2007) 24. Dehmer, M., Mowshowitz, A.: A history of graph entropy measures. Inform. Sci. 181(1), 57–78 (2011) 25. Dellschaft, K., Staab, S.: An epistemic dynamic model for tagging systems. In: Hypertext 2008, Proceedings of the 19th ACM Conference on Hypertext and Hypermedia, June 19–21, 2008, Pittsburgh, Pennsylvania, USA, 2008 26. Estrada, E.: Protein bipartivity and essentiality in the yeast protein-protein interaction network. J. Proteome Res. 5(9), 2177–2184 (2006) 27. Estrada, E., Rodr´ıguez-Velázquez, J.A.: Spectral measures of bipartivity in complex networks. Phys. Rev. E 72(4), 046105 (2005) 28. Fellbaum, C., (ed.): WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998) 29. Firth, J.R.: A synopsis of linguistic theory, 1933–1955. In: Firth, J.R., (ed.) Studies in Linguistic Analysis, pp. 1–32. Blackwell, Oxford (1957) 30. Freyd, J.J.: Shareability: The social psychology of epistemology. Cognit. Sci. 7, 191–210 (1983) 31. Hammwöhner, R.: Interlingual aspects of Wikipedia’s quality. In: Proceedings of the International Conference On Information Qualiy (ICIQ 2007) (2007) 32. Harary, F.: Graph Theory. Addison Wesley, Boston (1969) 33. Hollan, J., Hutchins, E., Kirsh, D.: Distributed cognition: toward a new foundation for humancomputer interaction research. ACM Trans. Comput. Hum. Interact. 7(2), 174–196 (2000) 34. Holme, P., Liljeros, F., Edling, C.R., Kim, B.J.: On network bipartivity. Phys. Rev. E 68, 056107 (2003)


317

35. Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: BibSonomy: A social book-mark and publication sharing system. In: Proceedings Of the Workshop on Tool Interoperability at the International Conference on Conceptual Structures 2006, pp. 87–102 (2006) 36. Hotho, A., Nürnberger, A., Paaß, G.: A brief survey of text mining. J. Lang. Tech. Comput. Ling. 20(1), 19–62 (2005) 37. Jäschke, R., Hotho, A., Schmitz, C., Ganter, B., Stumme, G.: Discovering shared conceptualizations in folksonomies. Web Semant. Sci. Serv. Agents World Wide Web 6(1), 38–53 (2008) 38. Klein, D.J., Ivanciuc, O.: Graph cyclicity, excess conductance, and resistance deficit. J. Math. Chem. 30(3), 271–287 (2001) 39. Köhler, R.: Systems theoretical linguistics. Theor. Ling. 14(2/3), 241–257 (1987) 40. Köhler, R.: Syntactic structures, properties and interrelations. J. Quant. Ling. 6, 46–57 (1999) 41. Koschützki, D., Lehmann, K.A., Peeters, L., Richter, S., Tenfelde-Podehl, D., Zlotowski, O.: Centrality indices. In: Brandes, U., Erlebach, T., (eds.) Network Analysis, vol. 3418, Lecture Notes in Computer Science, pp. 16–61. Springer, Berlin (2004) 42. Kunze, C., Lemnitzer, L.: GermaNet – representation, visualization, application. In Rodriguez, M., González, Paz Suárez Araujo, C., (eds.) Proceedings of the 3rd International Conference on Language Resources and Evaluation (LREC 2002), pp. 1485–1491. European Language Resources Association, Paris (2002) 43. Lambiotte, R., Ausloos, M.: Collaborative tagging as a tripartite network. In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J., (eds.) International Conference on Computational Science (3), vol. 3993, Lecture Notes in Computer Science, pp. 1114–1117. Springer, Berlin (2006) 44. Lenat, D.B.: CYC: A large-scale investment in knowledge infrastructure. Comm. ACM 38, 33–38 (1995) 45. Medelyan, O., Witten, I.H., Milne, D.: Topic indexing with Wikipedia. In: Proceedings of AAAI 2008 Workshop on Wikipedia and Artificial Intelligence: An Evolving Synergy (WikiAI08), Chicago (2008) 46. Mehler, A.: Text linkage in the wiki medium – a comparative study. In: Karlgren, J., (ed.) Proceedings of the EACL Workshop on New Text – Wikis and blogs and other dynamic text sources, pp. 1–8. Trento, Italy (2006) 47. Mehler, A.: Large text networks as an object of corpus linguistic studies. In: Lüdeling, A., Kytö, M., (eds.) Corpus Linguistics. An International Handbook of the Science of Language and Society, pp. 328–382. De Gruyter, Berlin/NewYork (2008) 48. Mehler, A.: On the impact of community structure on self-organizing lexical networks. In: Smith, A.D.M., Smith, K., Ferrer i Cancho, R., (eds.) Proceedings of the 7th Evolution of Language Conference (Evolang7), pp. 227–234. World Scientific, Barcelona (2008) 49. Mehler, A.: Structural similarities of complex networks: A computational model by example of wiki graphs. Appl. Artif. Intell. 22(7&8), 619–683 (2008) 50. Mehler, A.: Generalized shortest paths trees: A novel graph class applied to semiotic networks. In: Dehmer, M., Emmert-Streib, F., (eds.) Analysis of Complex Networks: From Biology to Linguistics, pp. 175–220. Wiley-VCH, Weinheim (2009) 51. Mehler, A.: Minimum spanning Markovian trees: Introducing context-sensitivity into the generation of spanning trees. In: Dehmer, M., (ed.) Structural Analysis of Complex Networks, pp. 381–401. Birkhäuser/Basel (2010) 52. Mehler, A., Geibel, P., Pustylnikov, O.: Structural classifiers of text types: Towards a novel model of text representation. J. Lang. Tech. Comput. Ling. 22(2), 51–66 (2007) 53. Mehler, A., Gleim, R., Ernst, A., Waltinger, U.: WikiDB: Building interoperable wiki-based knowledge resources for semantic databases. Sprache und Datenverarbeitung Int. J. Lang. Data Process. 32(1), 47–70 (2008) 54. Mehler, A., Gleim, R., Wegner, A.: Structural uncertainty of hypertext types. An empirical study. In: Proceedings of the Workshop “Towards Genre-Enabled Search Engines: The Impact of NLP”, in conjunction with RANLP 2007, pp. 13–19. Borovets, Bulgaria (2007) 55. Meluk, I.: Dependency Syntax: Theory and Practice. SUNY, Albany (1988)

318

A. Mehler

56. Mika, P.: Ontologies are us: A unified model of social networks and semantics. J. Web Semant. 5(1), 5–15 (2007) 57. Mika, P., Gangemi, A.: Descriptions of social relations. In: Proceedings of the 1st Workshop on Friend of a Friend, Social Networking and the (Semantic) Web (2004) 58. Milne, D., Witten, I.H.: An effective, low-cost measure of semantic relatedness obtained from Wikipedia links. In: Proceedings of AAAI 2008 Workshop on Wikipedia and Artificial Intelligence: An Evolving Synergy (WikiAI08), Chicago (2008) 59. Naranan, S., Balasubrahmanyan, V.K.: Models for power law relations in linguistics and information science. J. Quant. Ling. 5(1-2), 35–61 (1998) 60. Nelson, S.J., Johnston, W.D., Humphreys, B.L.: Relationships in medical subject headings. In: Bean, C.A., Green, R., (eds.) Relationships in the organization of knowledge, pp. 171–184. Kluwer Academic Publishers, New York (2001) 61. Newman, M.E.J.: The structure and function of complex networks. SIAM Rev. 45, 167–256 (2003) 62. Newman, M.E.J.: Power laws, Pareto distributions and Zipf’s law. Contemp. Phys. 46, 323–351 (2005) 63. Newman, M.E.J., Park, J.: The origin of degree correlations in the internet and other networks. Phys. Rev. E 68, 026121 (2003) 64. Niles, I., Pease, A.: Towards a standard upper ontology. In: Welty, C., Smith, B., (eds.) Proceedings of the 2nd International Conference on Formal Ontology in Information Systems (FOIS-2001), Ogunquit, Maine (2001) 65. Obdrˇza´ lek, J.: DAG-width: connectivity measure for directed graphs. In: SODA’06: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm, pp. 814–821. ACM, New York, NY, USA (2006) 66. OCLC. Dewey decimal classification summaries. A brief introduction to the Dewey Decimal Classification. http://www.oclc.org/dewey/resources/summaries/default.htm [accessed February 15, 2009], 2008 67. OpenCyc.org. OpenCyc documentation. http://www.opencyc.org/doc [accessed February 15, 2009], 2008 68. Pastor-Satorras, R., Vázquez, A., Vesipignani, A.: Dynamical and correlation properties of the internet. Phys. Rev. Letters 87(25), 268701 (2001) 69. Ponzetto, S., Strube, M.: Deriving a large scale taxonomy from Wikipedia. In: Proceedings of the 22nd National Conference on Artificial Intelligence (AAAI-07), pp. 1440–1447. Vancouver, B.C., Canada (2007) 70. Pustylnikov, O., Mehler, A.: Structural differentiate of text types. A quantitative model. In: Proceedings of the 31st Annual Conference of the German Classification Society on Data Analysis, Machine Learning, and Applications (GfKl), pp. 655–662 (2007) 71. Pustylnikov, O., Mehler, A.: Text classification by means of structural features. What kind of information about texts is captured by their structure? In: Proceedings of RUSSIR’08. Taganrog, Russia (2008) 72. Abramov, O., Mehler, A.: Automatic language classification by means of syntactic dependency networks. J. Quant. Ling. (2011) (accepted) 73. Ravasz, E., Barabási, A.-L.: Hierarchical organization in complex networks. Phys. Rev. E 67, 026112 (2003) 74. Rosch, E.: Principles of categorization. In: Rosch, E., Lloyd, B.B., (eds.) Cognition and Categorization, pp. 27–48. Erlbaum, Hillsdale, N.J. (1978) 75. Santini, M.: Characterizing genres of web pages: Genre hybridism and individualization. In: Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS’07) (2007) 76. Saunders, S.: Improved shortest path algorithms for nearly acyclic graphs. Ph.D thesis, University of Canterbury, Computer Science (2004) 77. Saunders, S., Takaoka, T.: Improved shortest path algorithms for nearly acyclic graphs. Theor. Comput. Sci. 293(3), 535–556 (2003)


319

78. Saunders, S., Takaoka, T.: Solving shortest paths efficiently on nearly acyclic directed graphs. Theor. Comput. Sci. 370(1-3), 94–109 (2007) 79. Searle, J.R.: Social ontology. Some basic principles. Anthropol. Theor. 6(1), 12–29 (2006) 80. Skorobogatov, V.A., Dobrynin, A.A.: Metrical analysis of graphs. MATCH 23, 105–155 (1988) 81. Sowa, J.F.: Knowledge Representation: Logical, Philosophical, and Computational Foundations. Brooks/Cole, Pacific Grove (2000) 82. Steels, L.: Collaborative tagging as distributed cognition. Pragmatics Cognit. 14(2), 287–292 (2006) 83. Steyvers, M., Tenenbaum, J.: The large-scale structure of semantic networks: Statistical analyses and a model of semantic growth. Cognit. Sci. 29(1), 41–78 (2005) 84. Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: WWW’07: Proceedings of the 16th international conference on World Wide Web, pp. 697–706. ACM, New York, NY, USA (2007) 85. Takaoka, T.: Shortest path algorithms for nearly acyclic directed graphs. Theor. Comput. Sci. 203(1), 143–150 (1998) 86. Tuldava, J.: Probleme und Methoden der quantitativ-systemischen Lexikologie. Wissenschaftlicher Verlag, Trier (1998) 87. Ulanowicz, R.E.: Identifying the structure of cycling in ecosystems. Math. Biosci. 65(2), 219–237 (1983) 88. Voss, J.: Collaborative thesaurus tagging the Wikipedia way. arXiv.org:cs/0604036 (2006) 89. Waltinger, U., Mehler, A., Heyer, G.: Towards automatic content tagging: Enhanced web services in digital libraries using lexical chaining. In: Cordeiro, J., Filipe, J., Hammoudi, S., (eds.) 4th Int. Conf. on Web Information Systems and Technologies (WEBIST ’08), pp. 231–236. INSTICC Press, Barcelona, Funchal, Portugal (2008) 90. Watts, D.J.: Six Degrees. The Science of a Connected Age. W. W. Norton & Company, New York/London (2003) 91. Zelinka, B.: Nearly acyclic digraphs. Czech. Math. J. 33(1), 164–165 (1983) 92. Zipf, G.K.: Human Behavior and the Principle of Least Effort. An Introduction to Human Ecology. Hafner Publishing Company, New York (1972) 93. Zlatic, V., Bozicevic, M., Stefancic, H., Domazet, M.: Wikipedias: Collaborative web-based encyclopedias as complex networks. Phys. Rev. E 74, 016115 (2006)

Chapter 11

Typology by Means of Language Networks: Applying Information Theoretic Measures to Morphological Derivation Networks Olga Abramov and Tatiana Lokot

Abstract In this chapter we present a network theoretic approach to linguistics. In particular, we introduce a network model of derivational morphology in languages. We focus on suffixation as a mechanism to derive new words from existing ones. We induce networks of natural language data consisting of words, derivation suffixes and parts of speech (PoS) as well as the relations between them. Measuring the entropy of these networks by means of so called information functionals we aim at capturing the variation between typologically different languages. In this way, we rely on the work of Dehmer (Appl Math Comput 201:82–94, 2008) who has introduced a framework for measuring the entropy of graphs. In addition, we compare several entropy measures recently presented for graphs. We check whether these measures allow us to distinguish between language networks on the one hand, and random networks on the other. We found out, that linguistic variation among languages can be captured by investigating the topology of the underlying networks. Further, information functionals based on distributions of topological properties turned out to be better discriminators than those that are based on properties of single vertices. Keywords Derivational morphology • Information functionals • Information theory • Network analysis MSC2010 Primary 94C15; Secondary 90C35, 05C90.

O. Abramov () University of Bielefeld, Universitätsstraße 25, 33615 Bielefeld, Germany e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 11, © Springer Science+Business Media, LLC 2011

321

322

O. Abramov and T. Lokot

1 Introduction Network models have gained importance in the humanities in recent years. A newly emerging branch of interdisciplinary research interest complements linguistic studies with methods in network theory. Following [22], a language is a complex dynamic system built of highly structured components (linguistic levels like syntax, morphology, etc.) that influence each other and are influenced by other linguistic and nonlinguistic factors. Standard approaches in linguistics enable to precisely describe single linguistic phenomena or cross-linguistic patterns (language universals) leaving aside the complex interactions between linguistic units in total. Quantitative linguistics (see, e.g., [1, 22]) bridges between linguistic phenomena, on the one hand, and their relations, on the other hand, using quantitative methods. Network models of language were recently discovered as an appropriate means to study the organization principles of language quantitatively [18, 28]. The reason is that networks allow to represent complex relations between linguistic units allowing to “zoom in,” and inspect their regularities. Ferrer i Cancho et al. [17] have shown that syntactic dependency networks of six different languages exhibit the same small-world (see, e.g., [39]) property, which seems to be universal for languages. Mehler [26] found out that ontological and functional differences of Wiki-networks can be recovered examining their topology. Pustylnikov and Mehler [33] used the same network model to cluster 11 languages into 3 genetic groups. Liu [25] could distinguish among genres within a single language comparing syntactic dependency networks of the same language. Mehler et al. [28] presented a novel approach to the Sapir–Whorf Hypothesis analyzing networks based on social ontologies of Wikipedia. Mehler et al. [29] introduced a network model of dialogue based on lexicons of communication partners, and demonstrated its potential for predicting lexical alignment of interlocutors. These and other studies on networks suggest that we can enhance linguistic research by means of network analysis. The study presented in this chapter aims to complement this field of research by presenting a network model for derivational morphology. We apply information theoretic measures to study the properties of morphological derivation networks (MDNs) compared to random graphs. The goal of the chapter is twofold: to motivate the use of network models in linguistics, and to evaluate some information theoretic measures for different kinds of graphs. We proceed as follows: • In particular, we focus on suffixation as one mechanism to derive new words from existing ones. We construct a network from words, derivation suffixes, and parts of speech (PoS) as well as the relations between them. • Measuring the entropy of these networks by means of so-called information functionals (see Sect. 3 for definition), we aim to capture the variation between typologically different languages. In this way, we rely on the work of [12] who has recently introduced a framework for the measurement of the entropy of graphs.

11 Typology by Means of Language Networks: Applying Information Theoretic...

323

• Furthermore, we aim to check whether the entropy measures allow us to distinguish between language networks, on the one hand, and random networks, on the other hand. In so doing, we rely on the work of [26, 27], that is, Quantitative Network Analysis (QNA) as a framework of network classification. To the best of our knowledge, this is the first empirically founded network model of morphology. It brings together two research branches, that is, QNA and graph entropy measurement, to shed light on an area of linguistic networking whose cognitive relevance has recently been claimed by [10]. Section 2 explains how the networks were induced, gives their formal definition, and discusses some of their properties. In Sect. 3, we present and discuss different graph entropy measures based on information functionals. In this section, we provide some modifications on the approach of [12] adapting it for our purpose. We evaluate these functionals on some characteristic example graphs (Sect. 4). In Sect. 5, we compute these functionals for language networks and compare them to random graphs. We cluster the graphs based on the values of the functionals. The results show that language networks can be perfectly distinguished from the random ones using the functionals. In Sect. 6, we discuss the results. The final conclusions are drawn in Sect. 7.

2 Morphological Derivation Networks The notion of the MDN introduced here is attributed to the organization of linguistic units in the area of derivational morphology. In this section, we describe how the networks were obtained (Sect. 2.1) and present their formal definition (Sect. 2.2).

2.1 Decomposition of Productive Suffixes In this section, we briefly discuss the decomposition algorithm (henceforth referred to as the decomposition algorithm DA) developed in [32] that is responsible for the induction of derivation rules from lexical input. The underlying theoretical framework behind DA is based on models of morphological processing (see [7, 11, 14]). In these models, suffixes that have the same function (e.g., to derive an adjective from noun) are supposed to compete during the evolution of language. For example, a suffix that is preferably used to derive an adjective from noun is likely to be reused in future word formations (e.g., fruit > fruit-ful). These suffixes are called productive suffixes of a language [3]. Morphological productivity has been thoroughly discussed in linguistic literature [2, 30, 36]. A lot of recent work was done to study the productivity of suffixes [6, 20, 31], and to measure it using quantitative approaches [3, 4, 16, 34, 35]. DA detects derivation suffixes in a language (if such exist) decomposing words into suffixes and stems. Pustylnikov and Schneider-Wiejowski [34] could show

324


that DA is able to identify productive suffixes in German analyzing texts from different periods of time (seventeenth to nineteenth century vs. twentieth century) and different registers (i.e., spoken vs. written). In this chapter, we construct the MDNs using the output of DA (i.e., suffixes and stems). The procedure underlying DA can be described as follows: DA parses texts that are pretagged with Parts-Of-Speech (PoS) information, that is, where the word category of each word is given. Four filtering steps are applied to filter out the derivation suffixes used in the input language. Roughly speaking, suffixes found in combination with a large number of different stems forming a particular PoS are considered to be significant in language. Doing so, the ten most significant suffixes are detected for each PoS which can be best used to form new words. These suffixes (as well as the corresponding stems) are taken to construct the MDNs.

2.2 Network Definition In the previous section, we outlined the decomposition algorithm (DA) which is used here to induce the derivation networks. Formally, the MDNs are multi-level graphs (see [26]) partitioned into three disjunct subsets of vertices, i.e., three-level graphs. Definition 1. Let G D .V; E/ be a graph of vertices V and edges E. We call G a three-level graph if the set V is represented as a union of three non-overlapping subsets, that is, V D V1 [ V2 [ V3 with Vi \ Vj D ;; 8i; j D 1; 2; 3 ^ i ¤ j . There can exist edges between and within the three subsets of V . The vertex subsets of V are obtained from three different sources that are described below. Instead of speaking about the subsets V1 ; V2 , and V3 , we will speak about W; S , and P respectively, which are explained in the following: 1. Vertices belonging to the first subset W are words and stems obtained from the lexical input. 2. The second subset S contains significant suffixes identified via decomposition (see the previous section). 3. The last subset P includes PoS. Note that Definition 1 differs from the definition of a k-partite graph, which does not allow for edges between vertices within a subset Vi . In our case, edges can occur between different subsets (e.g., between S and P ) as well as within the set W . No edges occur among vertices of the subsets S and P . These observations restrict the connectivity of the graph G (see also Fig. 11.1 for illustration) in the following way: 1. If v 2 W; w 2 V an edge hv; wi 2 E can exist. 2. If fv; wg 2 S ! hv; wi … E. 3. If fv; wg 2 P ! hv; wi … E.


325

Fig. 11.1 An example MDN. Subsets W D words and stems, S D suffixes, and P D PoS

This network model allows us to map the morphological structure of different languages (i.e., English vs. German and vs. a language without derivational morphology). For example, in a language that does not use suffixes, the second level S will be missing.

2.3 Data: Networks and Their Topological Properties In this chapter, we compare three kinds of graphs: 1. Morphological Derivation Networks: • English • German • Random Words MDN (RMDN) 2. Random graphs: • Erd˝os and Rényi [15] graphs (ERN) 3. Small world graphs: • Scale-free graphs [5] • Small world graphs [39] The MDNs were constructed based on word lists of the same size that were decomposed into stems and suffixes using DA. ERN are randomly generated networks with cardinalities of German and English MDNs. The RMDN was constructed from

326


Table 11.1 Network characteristics: number of nodes, edges, self loops, connected components, diameter, radius, average shortest path length, clustering coefficient [39], density [38], heterogeneity [38], centralization [19], average degree, gamma and the coefficient of determination of the power law fit of degrees, average betweenness centrality [9], gamma and the coefficient of determination of the power law fit of BCs Feature German English RMDN ERN 195 ERN 163 BA 195 WS 195 #Nodes 195 163 136 195 163 195 195 #Edges 297 255 134 939 611 387 585 #Self loops 4 22 0 10 5 0 128 #Connected 1 1 2 1 1 1 1 component Diameter 7 7 7 5 5 6 7 Radius 4 4 1 3 4 4 5 Average 3:66 3:68 3:99 2:58 2:76 3:31 3:84 shortest path length Clustering 0:01 0:08 0 0:04 0:05 0:087 0:15 coefficient Density 0:015 0:018 0:015 0:04 0:04 0:02 0:02 Heterogeneity 1:32 1:34 3:08 0:3 0:3 1:08 0:27 Centralization 0:19 0:21 0:354 0:04 0:03 0:17 0:02 1:51 1:49 0:98 10:26 11:64 1:98 3 Average degree .degree/ 1:91 1:98 3:09 0:63 0:61 1:6 0:06 R2 .degree/ 0:98 0:99 0:99 0:69 0:81 0:99 0:96 Average BC 0:014 0:037 0:02 0:008 0:01 0:02 0:02 .BC / 0:99 2:29 1:01 0:35 0:39 0:85 0:05 R2 .BC / 0:99 0:99 0:77 0:84 0:81 0:96 0:99

a randomly generated word list of nonsense words that consisted of letters from the Latin alphabet. This word list served as input to DA. The DA tried to induce significant derivation suffixes from random words. Of course, the nonsense words did not have any internal structure, though, some suffixes occurring due to chance in more than one words were detected. This fact explains some coincidental links between the noun and verb subgraphs in Fig. 11.7. As can be seen, single words and suffixes group around the three PoS categories without any additional organization within the word families. The English and German MDNs are displayed in Figs. 11.5 and 11.6. Table 11.1 lists some of their topological characteristics. In the following, we like to draw attention to some of their properties. When we compare the connectivity of both language networks to RMDN, we see that the former two have many more edges than nodes. The proportion of nodes to edges is also similar for both natural networks. The RMDN has more vertices than edges and two disconnected components. This can be explained by a very small number of common parts (i.e., stems, suffixes) in the random vocabulary. That is, random words have a low average degree (0.98) by connecting to PoS vertices.1 1

In fact, only 4 nodes of 136 have a degree > 1.


327

The RMDN has three parts of vertices grouping together (star-like) around the three main PoS. The noun and verb parts are connected together by a coincidental link, adjectives constitute a separate component. All in all, the star-graph-like form of the RMDN is also confirmed by the higher centralization [19] (3.5 vs. 2) and heterogeneity [38] (3.08 vs. 1:4) values in contrast to natural networks. Density [38] is a parameter that indicates the average number of neighbors. Density ranges from Œ0; 1 and shows how densely the vertices of a graph are connected. For all the MDNs, the density values are comparably small as are the values of the clustering coefficient [39]. This can be explained by a selective connectivity among the three levels of the multi-level graph, as explained in Sect. 2.2. In all MDNs, vertices of the set P do not link together but link to the vertices of S and W . The same holds for suffixes, which do not connect together but connect only to words and PoS. Words connect to other words only if they belong to the same word family (e.g., geography > geographical > etc.) These peculiarities lower the probability of common neighbors in the MDN, for random as well as natural networks. Note that English has a slightly larger density value (0.018) and a slightly higher clustering coefficient (0.08) than the other networks. This is probably because English has less morphological variety within the word that leads to a multi-functionality of items in W , and raises their probability to be connected to each other. If the words in W would be completely random, the lack of significant suffixes would lead to a structure comparable to RMDN. However, English words are not just random combinations of letters that results in a redundancy of the same words in several word classes and a higher connectedness among them. A remarkable property is the number of self loops showing different values for all the three MDNs. English has a much higher number of self loops than German and RMDN (22 > 4 > 0). This property also points to the fact that the same word forms in English often function as different PoS and stems. For example, the word lift can occur as a stem, verb, and noun. English contains many similar examples; thus, the number of self loops discriminate English from other networks according to its morphological property. In summary, the MDNs of natural languages can be distinguished from the random MDNs when comparing their topological properties. Furthermore, differences in morphological structure among natural languages become visible by looking at the topology of MDNs. In Figs. 11.5, 11.6 and 11.7, the different MDNs are visualized by their centrality values (i.e., the vertex with the highest centrality is the biggest one). In English and in RMDN, the central vertices are nouns, verbs, and adjectives. In German, in contrast, the most central are nouns, the suffix -en together, and adjectives. In this network, -en is more central than the PoS VV (“verb”) because this suffix is attached not only to almost all the verbs in German infinitive, but also to a large number of nouns (sehen vs. das Sehen). This example illustrates how morphological differences of a particular language form the topology of the network.

328


3 Measuring the Entropy of MDNs Numerous measures for analyzing complex networks quantitatively have been contributed [8]. Always, the starting point is to infer structural characteristics from a graph under consideration which take its structural features into account. Among the existing network complexity measures, there exist two main categories [8]: Non-information-theoretic and information-theoretic measures. The latter indices have intensely been used for characterizing graphs in mathematical chemistry which rely on Shannon’s entropy [37]. Most of the existing information-theoretic network measures represent so-called partition-based measures, i.e., we group certain elements of a graph invariant (e.g., vertices, edges, distances, degrees, etc.) into equivalence classes and assign a probability value to each such class. In contrast, Dehmer [12, 13] developed another approach to avoid the problem of determining the just mentioned partitions. In [12, 13], a probability value has been assigned to each individual vertex using an information functional [12,13] that leads to families of graph entropy measures. In this section, we present and discuss several information functionals that we evaluate for our networks.

3.1 Graph Entropy by Means of Information Functionals We build on the approach of graph entropy measurement as developed by [12]. The reason is that other than related approaches to graph entropy measurement, [12] relies on assigning a probability values to each vertex of a graph instead of determining partitions, involving a graph invariant, that might be computationally costly. This is what we need when evaluating the structural role of single morphemes in a morphological network. This section recapitulates basic notions of graph entropy measurement as introduced by [12]. This holds for Definitions 2–6, which basically repeat the corresponding framework of [12]. We complement several propositions as well as a lemma on information functionals as the basic building blocks of entropy measurement. This will be the starting point of measuring the entropy of morphological networks as proposed in the following sections. Following Definition 2.7 in [12], if we have a finite undirected graph G D .V; E/ with V being the set of vertices and E the set of edges, and we have a positive function f on the set V (called an information functional in [12]), then we can define: f .vi / p.vi / D PjV j : (11.1) kD1 f .vk / Since the equality p.v1 / C p.v2 / C : : : C p.vjV j / D 1 holds, we can interpret the values p.vi / (i D 1; 2; : : : ; jV j) as vertex probabilities.


329

Having such a probability distribution, we immediately compute the entropy If .v/ of G, which is interpreted here as the mean structural information content.2 If .G/ D

jV j X

p.vi / ln p.vi /:

(11.2)

i D1

Dehmer [12] presents some novel information functionals of V which capture, in some sense, the structural information of the underlying graph G. We concentrate on the functional f V and prove some statements (in Propositions 1–3, Lemma 1), which allow us to simplify the use of f V for our purpose. We first need to repeat some preliminary definitions as well as the definitions of the information functional f V given in [12]. Note that the length of a path on the graph G is measured (in [12] and here) as a number of edges in this path. We denote 8u; v 2 V the length of the shortest path between them by d.u; v/. Definition 2. The quantity D .G/ WD maxu;v2V d.u; v/ is called the diameter of G. Definition 3. The set Sj .vi ; G/ WD fv 2 V jd.vi ; v/ D j; j 1g is called the j -sphere of vi regarding G. Definition 4. Given a vertex vi 2 V and the j-sphere Sj .vi ; G/, according to [12], we define the local information graph LG .vi ; j / as follows: for all w 2 Sj .vi ; G/ the shortest path connecting w and vi has the length j by definition of Sj .vi ; G/. There is not necessarily only one such path for the vertex w, but we take only one path of the length j for every w 2 Sj .vi ; G/. Then, these paths with their edges and vertices form a subgraph of G which is called the local information graph and denoted by LG .vi ; j /. j is called the local information radius regarding vi . Now we formulate and prove a lemma which shows that 8vi 2 V all j -spheres Sj .vi ; G/ with j D 1; 2; : : : ; cover the set V n fvi g and 8j; k with 1 j; k and k ¤ j the equality Sj .vi ; G/ \ Sk .vi ; G/ D ; holds. Lemma 1. Let P G D .V; E/ be a finite undirected connected graph. Then, 8vi 2 V the equality j D1 jSj .vi ; G/j D jV j 1 holds where jV j .jSj .vi ; G/j/ is the cardinality of the set V .Sj .vi ; G//, respectively, and D .G/ is the diameter of G as defined above. Proof. Let vi be an arbitrary vertex on G. We show that 8w 2 V n fvi g there exists a j-sphere Sj .vi ; G/ on which the vertex lies. Indeed, if we take j D d.w; vi /, then, obviously, w 2 Sj .vi ; G/. Furthermore, if we have two natural numbers k and j with k ¤ j and j k; j , we can easily see that

2 Dehmer [12] uses log to calculate the entropy. We use ln here for all functionals, which does not have any impact on the final results of the relative entropy (see the definition below) values.

330


Sj .vi ; G/ \ Sk .vi ; G/ D ;: These two observations complete the proof of Lemma 1. Definition 5. Given a local information graph LG .vi ; j / regarding vi 2 V , we denote (see [12]) the sum of the lengths of all shortest paths in LG .vi ; j / selected in Definition 4, each of which connects vi with some point of Sj .vi ; G/ by l.P .LG .vi ; j ///. It was proved in [12] (see Proposition 3.1) that 8vi 2 V and j D 1; 2; : : : ; it holds: l.P .LG .vi ; j /// D j jSj .vi ; G/j:

(11.3)

We are now able to present the definition of the information functional f V introduced in [12]. Definition 6. The information functional f V is defined 8vi 2 V by the formula: f V .vi / WD ˛

P

j D1 cj jSj .vi ;G/j

;

(11.4)

where cj with j D 1; 2; : : : ; and ˛ are arbitrary real positive parameters. In the following, we prove some properties of f V . Proposition 1. If c1 D c2 D : : : D c > 0, then 8˛ > 0; 8vi ; vj 2 V , we have f V .vi / D f V .vj /:

(11.5)

Proof. In view of our assumptions and Lemma 1 8vi 2 V , we have: f V .vi / D ˛

P


P

D ˛ c1

j D1

jSj .vi ;G/j

D ˛ c1 .jV j1/ : Hence, the value f V .vi / does not depend on the vertex vi . Corollary 1. If c1 D c2 D : : : D c > 0, then for the probability distribution p V on V induced by the information functional f V with the formula (11.1), 8˛ > 0; 8vi ; vj 2 V we have p V .vi / D p V .vj /. The corresponding entropy has the maximal possible value for G which equals ln jV j where jV j is the cardinality of the vertex set V . Proof. In view of (11.1) and Proposition 1 8vi ; vj 2 V we immediately obtain: p V .vi / D p V .vj / D

1 ; jV j


331

and for the entropy If V .G/ we get: jV j X 1 1 If V .G/ D ln jV j jV j i D1 D ln.jV j/: Proposition 2. Given a set of positive parameters ˛; c1 ; c2 ; : : : ; c for the information functional f V and an arbitrary positive number c, we consider another set of parameters c1 C c; c2 C c; : : : ; c C c with the same ˛. Then the probability distribution and hence the entropy for both parameter sets are equal. Proof. To prove the statement of Proposition 2, we denote 8vi 2 V the values of f V with parameters c1 ; c2 ; : : : ; c by f V .vi ; c1 ; : : : ; c /. We have: f V .vi ; c1 C c; c2 C c; : : : ; c C c/ D ˛ D˛

P

j D1 .cj Cc/jSj .vi ;G/j

P

P

j D1 cj jSj .vi ;G/jCc

j D1 jSj .vi ;G/j

:

In view of Lemma 1, we obtain: f V .vi ; c1 C c; c2 C c; : : : ; c C c/ D f V .vi ; c1 ; c2 ; : : : ; c / ˛ c.jV j1/ : Thus, we can see that if we add a constant c > 0 to each cj ; j D 1; 2; : : : ; 8vi 2 V , the new value of f V will be a product of the old value and the constant ˛ c.jV j1/ . As easily follows from (11.1), the corresponding probability distribution and hence the entropy value does not change. Proposition 3. If ˛ > 1 (˛ < 1) for f V we can set ˛ D 2 .˛ D 12 /, respectively, without loss of generality. Proof. Given a set of positive parameters ˛ > 1; c1 ; c2 ; : : : ; c , we can choose the parameters c10 ; c20 ; : : : ; c0 , so that 8vi 2 V ˛

P


D2

P

0 j D1 cj jSj .vi ;G/j

;

holds. Indeed, if we put cj0 D cj log2 ˛; j D 1; 2; : : : ; , then we get 2

P

0 j D1 cj jSj .vi ;G/j

D 2. D˛

P

j D1 cj jSj .vi ;G/j/log2

P


˛

:

Hence, we see that if we have a set ˛; c1 ; c2 ; : : : ; c of positive parameters with ˛ > 1 and consider the other set of positive parameters 2; c1 log2 ˛; c2 log2 ˛; : : : ; c log2 ˛, then 8vi 2 V the value of the information functional f V does not change.

332


Remark 1. Proposition 3 shows that we can reduce the number of parameters for the information functional f V by taking ˛ D 2 (or ˛ D 12 ). Hence, f V can be now defined by the following equation: f V .vi / WD 2

P


; 8vi 2 V;

(11.6)

with c1 ; c2 ; : : : ; c being positive parameters. Moreover, if we consider the numbers c1 ; c2 ; : : : ; c being simultaneously positive or negative, we can cover the both cases ˛ D 2 and ˛ D 12 with the formula above. For short, we can treat the set c1 ; c2 ; : : : ; c of parameters as a -dimensional vector cN D fc1 ; c2 ; : : : ; c g and the set jS1 .vi ; G/j,jS2 .vi ; G/j; : : : ; jS .vi ; G/j 8vi 2 V as the vector function SN .vi / D fjS1 .viP ; G/j; jS2 .vi ; G/j; : : : ; j S .vi ; G/jg. Furthermore, instead of dealing with the sum j D1 cj jSj .vi ; G/j, we can write the scalar product (c; N SN .vi // of two -dimensional vectors cN and SN .vi /, which is simply the sum c1 jS1 .vi ; G/j C c2 jS2 .vi ; G/j C : : : C c jS .vi ; G/j. The formula that defines f V can be given now as follows [13]: N

N WD 2.c;N S.vi // ; f V .vi / D f V .vi ; c/

(11.7)

where the coordinates of the -dimensional vector cN can be taken all positive or all negative. Hence, we see that it suffices to use the functional f V varying only one set of parameters fc1 ; c2 ; : : : ; c g without loss of information. Remark 2. For simplicity, [13] considers only the exponent in the formula 11.7. Hence, instead of using (11.4) we can take the functional f V as follows: f V .vi / D .c; N SN .vi //:

(11.8)

In the following, we will use this version of the functional when computing the entropy of the graphs.

3.2 Information Functional on the Set J D f1; 2 ; : : : ; g In this section, we present the information functional f J [23] that is actually a function on the set J D f1; 2; : : : ; g with being the diameter of the graph G D .V; E/. Definition 7. Using the Definition 3 of a j-sphere, we define a function f J on the set J as follows: f J .j / WD

jV j X i D1

jSj .vi ; G/j:

(11.9)


333

The value of f J .j / gives the sum of the cardinalities of all j -spheres in G. The probability p J .j / for j can be calculated by the standard formula: f J .j / : p J .j / D P J i D1 f .i /

(11.10)

Hence, the entropy of G based on f J can be calculated as usual according to the well-known formula of entropy as it was shown in the previous section (see also [12]). X J p J .j / ln p J .j / : (11.11) If .G/ D j D1

In addition, we calculate the relative entropy of a graph given by the formula: I.f J / INfJ .G/ D 2 Œ0; 1; ln

(11.12)

for > 1. For the functional f V , respectively, we calculate I.f V / 2 Œ0; 1; INfV .G/ D ln jV j

(11.13)

with jV j being the number of vertices in the graph.

3.3 Information Functional Based on Distances The next measure, we like to present is the information functional based on distances in the graph proposed in [24] and evaluated in [23] on molecular graphs. The results in [23] state that this functional distinguishes well between polycyclic graphs and trees. Graphs studied in [23] are small graphs representing molecular structures. In this chapter, we test the ability of this functional to discriminate between more complex networks. The information functional proposed in [23] is calculated for a vertex vi as the entropy of its shortest distances from all other vertices in the graph: HD .vi / D

X d.vi ; u/ u2V

D.vi /

ln

d.vi ; u/ ; D.vi /

(11.14)

P with D.vi / D u2V d.vi ; u/. The aggregation function over all distances of vertices in the graph is proposed in [23] as follows: HDV D

X v2V

HD .v/:

(11.15)

334


The codomain of this function does not lie within the interval of Œ0; 1, which is preferable to compare the graphs. Instead of normalizing the above function, we use the function D.vi / as an information functional: f D .vi / WD D.vi / D

X

d.vi ; u/:

(11.16)

u2V

Then, the corresponding probability p D is given by the formula: p D .vi / D P

f D .vi / : D v2V f .v/

(11.17)

Given these probabilities, the entropy and the relative entropy can be calculated straightforward: X p D .v/ ln p D .v/ (11.18) IfD .G/ D v2V

I.f D / INfD .G/ D 2 Œ0; 1: ln jV j

(11.19)

3.4 Information Functional Based on the Distribution of Distance Sums In addition to f D , we present an information functional based on the distribution of distance sums. The reason for introducing this measure is the rather unsatisfactory separability of the functional f D . Konstantinova [23] shows that f D possesses a high discriminative potential for distinguishing molecular graphs. However, molecular graphs used in [23] are rather small graphs of about 20 vertices. Applying the functional to more complex networks results in a poor performance (see Table 11.6 in Sect. 5). P The reason for that might be that the sum v2V f D .v/ in (11.17) produces some redundancy, since each distance sum f D .vi / contains jV j 1 other sums of other vertices (while the graph is connected). For our graphs, which are more complex than molecular graphs, the difference between single f D .vi / and the total sum P D D v2V f .v/ is always large, resulting in similarly small probabilities p .vi / and in an indistinctive measure of entropy (see Table 11.6). To overcome this problem, it was necessary to find another way to obtain the probabilities of distance sums. For that reason, we decided to explore the distribution of vertex sums by means of a new information functional. We consider the functional f DS on the set f1; 2; : : : ; Rg with R being the number of different values of the functional f D on G (see Equation 11.16), that is, we


335

enumerate somehow the different values of f D on G using the numbers 1; 2; : : : ; R. Hence, for each vertex v 2 V we get some number ind.v/ 2 f1; 2; : : : ; Rg that equals to the number the value f D .v/ has got by our enumeration. Thus, for any v; u 2 V the equality ind.v/ D ind.u/ holds iff f D .v/ D f D .u/. The functional f DS can be defined as follows: f DS .k/ WD jfvjv 2 V; k D ind.v/gj:

(11.20)

Now, the probability for each k 2 f1; 2; : : : ; Rg can be defined straightforward: f DS .k/ f DS .k/ p DS .k/ D PR D : DS .i / jV j i D1 f

(11.21)

The entropy and the relative entropy are calculated by the following formulae: IfDS .G/ D

R X

p DS .k/ ln p DS .k/

(11.22)

kD1

I.f DS / INfDS .G/ D 2 Œ0; 1; ln R

(11.23)

for R > 1.

3.5 Information Functional Based on Betweenness Centralities At least, we calculate the entropy based on the distribution of betweenness centralities [9] (BC for short) of vertices in G. For each vertex v 2 V in G, we calculate first the value BC.v/ (see [9]). Let l be the number of different values of the function BC. Then, we enumerate somehow the different values of BC in G using the numbers 1; 2; : : : ; l. Hence, for each vertex v we get some index ind.v/ that equals the number the value BC.v/ has got by our enumeration. For any two vertices u and v, the equality ind.v/ D ind.u/ holds iff BC.v/ D BC.u/. Now we define the functional f BC on the set B D f1; 2; : : : ; lg 8k 2 B as follows: f BC .k/ WD jfvjv 2 V; k D ind.v/gj: (11.24) The probability for each k 2 B can be defined straightforward: f BC .k/ f BC .k/ p BC .k/ D Pl : D BC .i / jV j i D1 f

(11.25)

336


The corresponding entropy and the relative entropy can be calculated as follows: IfBC .G/ D

l X

p BC .k/ ln p BC .k/

(11.26)

kD1

I.f BC / 2 Œ0; 1; INfBC .G/ D ln l

(11.27)

for l > 1.

4 Evaluation 4.1 Applying Information Functionals to Example Graphs In this section, we present the entropy values calculated using the information functionals f V , f J , f D , f DS and f BC . To examine the behavior of the functionals, we selected some characteristic graphs of the same cardinality but differing in structure. These are a linear graph (a), a star graph (b), a tree graph (c), a complete graph (CG) (d), and a circular graph (e) (see Fig. 11.2). In addition, we calculate the entropy based on betweenness centralities of the graphs to compare the results with the outcomes produced by the functionals. Table 11.2 lists the relative entropy values of these graphs. The f D functional discriminates neither between linear and tree graphs (0.99), nor between CG and circular graphs (1.), the star graph is slightly different but almost undistinguishable from the tree and linear graphs (0.991 vs. 0.99). The functionals f V , f DS , and f J assign to the CGs the lowest entropy (0.). The circular graph has the highest entropy for all functionals except for f DS that has an opposite value (0 vs. 1). However, the f J rates the circular graph with a value (0.975) below 1. Increasing the entropy, the

Fig. 11.2 Example graphs of eight vertices: (a) linear graph, (b) star graph, (c) tree graph, (d) complete graph (CG) and (e) circular graph. The gure is taken from [26]


337

Table 11.2 Example graphs. The parameters used to calculate INfV are: ˛ D 0:5, c1 D , c2 D 1; : : : ; c D 1. The INfBC is calculated on the distribution of BCs in the graph in analogy to INfDS INfV INfD INfDS INfBC Graph INfJ jV j Linear graph Star graph CG Tree graph Circular graph

0.904 0.863 0 0.92 0.975

0.376 0.941 1 0.515 1

0.99 0.991 1 0.99 1

1 0.591 0 0.967 0

0.843 0 1 0.947 1

7 2 1 5 4

8 8 8 8 8

f J , f DS , and f BC give the star graph the lowest entropy value after the CG. Here again, the f J assigns a value above zero (0.863) to CG in contrast to the other two functionals. For f V , the linear graph has the lowest entropy followed by the tree and the star graph. The CGs have always the entropy of 1 irrespective of the parameters used. This is a nice property of f V that allows to immediately filter out the CGs of an arbitrary size. f DS assigns to CG and to circular graphs the same entropy of zero, indicating that these different types of graph are not distinguished by the distribution of distance sums. Other example graphs are ranked by f DS similar to the ranking of f J , that is, with an increase of entropy we get: IN(star graphs) < IN(tree graphs) < IN(linear graphs). According to these preliminary observations, the f J functional behaves similar to the f BC but has a higher discriminative potential. The f D shows a poor discriminative ability on the graphs studied here. The f DS performs better than f D but it does not distinguish the CG from circular graphs. The f V functional seems to weight the graphs differently resulting in values different from the other functionals, especially for star, linear, and tree graphs. Obviously, the functionals reflect different topological properties of the graphs. In the next sections, we will see how these functionals behave when applied to more complex networks.

4.2 Parameter Study for f V In this section, we look more closely at the functional f V experimenting with parameters. First, we discarded the parameter ˛, as suggested in Remark 2, considering only the sum of cardinalities of j-spheres with the corresponding coefficients. The resulting formula to compute the functional f V was chosen as follows: X V N f .vi / WD .c; N S .vi // D cj jSj .vi ; G/j: (11.28) j D1

We compared the results for different sets of parameters: 1. ./: c1 D 0; : : : ; c1 D 0; c D 1 2. . 2 /: c1 D 0; : : : ; cŒ D 1; : : : ; c D 0, for > 1 else I D 1 2

338

O. Abramov and T. Lokot Table 11.3 Relative entropy values for the example graphs using differ ent parameter combinations. ./: c1 D 0; : : : ; c1 D 0; c D 1, . 2 /: c1 D 0; : : : ; cŒ 2 D 1; : : : ; c D 0, .C/: c1 D 1; : : : ; c1 D 1; c D and ./: c1 D ; c2 D 1; : : : ; c D 1 Graph Star graph Circular graph CG Linear graph Tree graph

() 0.93 1 1 0.33 0.5

.2/ 0.8 1 1 1 0.97

() 0.99 1 1 0.99 0.99

(C) 0.99 1 1 0.99 0.99

3. .C/: c1 D 1; : : : ; c1 D 1; c D 4. ./: c1 D ; c2 D 1; : : : ; c D 1 In the first case, the j-sphere with j D D diameter was weighted by 1 and all other spheres by 0. In the second case, the “middle” j-sphere (i.e., j D 2 ) was weighted by 1 and the other j-spheres by 0. The last two alternatives weight the j-spheres by values from 1 to increasing (or decreasing) with j . Table 11.3 shows the results computed for the example graphs (see Fig. 11.2). We see from the table that star, circular, and CGs have the same values for all four parameter combinations. Linear and tree graphs, however, show a considerable difference for ./, on the one hand, and the other parameter combinations, on the other hand. That is, the same functional produces very different entropy values for the same graph (e.g., 0.33 vs. 1 for linear graphs) when we vary the parameter c. Nevertheless, ./ and to some extent . 2 / distinguish different types of graphs from each other, which was not the case for the other two combinations.

5 Results We use the parameters ./ and . 2 /3 to compute the entropy for English, German, and Random-Word MDNs (RMDN), as well as for random graphs like Erd˝os and Rényi graphs (ER), and scale-free graphs (BA, WS).4 The RMDN contains disconnected parts (see Fig. 11.7); thus, we calculate the entropy for the whole network as well as for the largest connected component (RMDNLCC ). Table 11.4 lists the resulting values calculated by means of f V .

3

We selected these combinations since they performed best in the parameter study shown in Table 11.6. 4 ER graphs are connected undirected random [15] graphs of the cardinality of German and English. BA [5] and WA [39] are randomly generated small world graphs of the cardinality of German. We generate ten graphs of each kind of random network (i.e., ten graphs for ER, ten for BA, etc.) and compare the averaged entropy values.


339

Table 11.4 Relative entropy values using two parameter combinations ./: c1 D 0; : : : ; c1 D 0; c D 1 and . 2 /: c1 D 0; : : : ; c 2 D 1; : : : ; c D 0. ER, BA, and WS graphs are presented in terms of average values and corresponding standard deviations (STD) Graph German English RMDNLCC ER ER BA WS

Fig. 11.3

() 0.46 0.53 0.8 0.882 0.519 0.600 0.475

STD – – – 0.253 0.273 0.078 0.188

.2/ 0.99 0.99 0.77 0.989 0.988 0.958 0.992

STD – – – 0.003 0.004 0.002 0.002

jV j 195 163 136 195 163 195 195

V Comparison of relative entropy values (f DS , f J , f./ and f.V / ) computed for ten 2

V are: c1 D 0; : : : ; c1 D 0; randomly generated ER 195 graphs. Sets of parameters used for f./ V c D 1 and for f. / : c1 D 0; : : : ; c 2 D 1; : : : ; c D 0 2

At first glance, the first parameter set ./ seems to produce more realistic results assigning lower values to German and English than to RMDNLCC , and distinguishing well between the single graphs. However, we observe high fluctuations between single graphs of the same type as becomes evident from the high standard deviations (STD) of about 0.2. Figure 11.3 illustrates how the entropy values vary for ER graphs of 163 vertices compared to the values of f DS , f J , and f.V / . 2 Furthermore, we look for the possibility to discriminate between the different types of networks by means of the information functionals discussed so far. We apply Quantitative Network Analysis (QNA) from [26, 27] to learn classes of

340


morphological networks by virtue of their structure, while disregarding any content units (i.e., names of vertices). QNA basically integrates vector representations of complex networks with hierarchical cluster analysis. The cluster analysis is complemented by a subsequent partitioning, where the number of classes is determined in advance. In this sense, QNA is semi-supervised [28]. The basic idea of QNA is to provide highly condensed numerical representations of networks that nevertheless capture their structural characteristics so that they can be automatically classified. In our framework, QNA works as follows: given a vector representation of each graph (with dimensions representing entropy values based on six information funcV tionals: f J , f D , f./ , f.V / , f DS , and f BC ) hierarchical clustering is applied. The 2 algorithm tries out several linkage methods (complete, single, average, weighted, centroid, median, ward) and several distance metrics (mahalanobis, correlation, euclidean, seuclidean, cityblock, jaccard, spearman, cosine) to find the best way to separate the data. We use F-Measure statistics5 to evaluate the classification. For the known partition of networks L and the partition found by the clustering algorithm P, the F-Measure is computed as follows: F-Measure.P; L/ D

X 2 Recall.P; L/ Precision.P; L/ 2 Œ0; 1: Recall.P; L/ C Precision.P; L/ L2L

(11.29)

For P 2 P being the number of networks classified to a group, and L 2 L the \Lg real number of networks belonging to this group, Precision D #fP 2 Œ0; 1 is #fP g the rate of correctly classified networks with respect to all networks classified to \Lg 2 Œ0; 1 is the rate of correctly classified networks a group. Recal l D #fP #fLg according to the total number of networks belonging to the group. The F-Measure ranges between 0 and 1. A value close to 1 indicates that networks were classified correctly with respect to their type, and a value nearby 0 shows that the classification failed. All entropy values used for classification are presented in Table 11.6.

6 Discussion Figure 11.4 and Table 11.5 present the results of classifying the graphs by means of six functionals. All types of networks could be separated perfectly (F-Measure of 1). German and English are clearly close to each other and can be distinguished by means of these measures. Considering the random graphs, the BA network is the most similar one to English and German.

5

See [21] for details.


341

Fig. 11.4 Clustering of graphs as feature vectors of six entropy values Table 11.5 Classification into five groups: (1) German and English, (2) ER, (3) RMDN, (4) WS, (5) BA F-Measure Random baseline Groups Clustering Linkage Best features V 1.0 0.71 5 Hierarchical Ward f./ ; f DS ; f BC V Table 11.6 Entropy measured using f J , f./ , f.V / , f D , f DS , and f BC 2

Graph

INfJ

INfV./

INfV

INfD

INfDS

INfBC

jV j

German English RMDN RMDNLCC ER ER BA WS

0.724 0.767 0.71 0.715 0.666 0.652 0.714 0.753

0.469 0.577 – 0.826 0.882 0.519 0.600 0.475

0.99 0.99 – 0.844 0.989 0.988 0.958 0.992

0.998 0.998 0.997 0.999 0.999 0.999 0.998 0.999

0.927 0.952 0.68 0.625 0.960 0.963 0.977 0.973

0.74 0.775 0.398 0.393 0.968 0.952 0.744 0.387

7 7 7 7 4 5 6 8

195 163 136 111 195 163 195 195

.2/

Furthermore, we performed a genetic feature selection study filtering out the redundant features that do not improve the result of classification. According to V this study, the best functionals are f./ , f DS , and f BC , which are all based on distributions of different topological properties of graphs. When we look more closely at the relative entropy values (Table 11.6), we see V that English has a slightly higher entropy than German according to f J , f./ , f DS , BC and f . This result is in accordance with what we would expect comparing the use

342


Fig. 11.5 German MDN. Visualization of the betweenness centralities. The three most central nodes: Noun, -en suffix, Adjective

Fig. 11.6 English MDN. Visualization of the betweenness centralities. The three most central nodes: Noun, Adjective, Verb

of derivational morphology in German and English. That is, German has a higher predictability of PoS by stems and suffixes than English. At the same time, the most functionals assign lower entropy values to RMDNLCC than to English and German, V except for f D and f./ , although, in case of f D the difference is very small. It seems more plausible to expect natural language networks to have lower entropies


343

Fig. 11.7 Random MDN. Visualization of the betweenness centralities. The three most central nodes: Noun, Verb, Adjective

V than random, like those obtained from f./ . However, the other functionals might take different properties of networks into account like, for example, the centrality of the graph. Regarding centrality, the RMDN is more centralized than the other two, which is visible from Fig. 11.7. The values of f BC confirm this fact assigning to RMDNs lower entropy values. The functional f D produces almost equal values for all graphs at the first and second decimal point (0.99). The functional f DS , which is based on distributions of distance sums, produces much better results.

7 Conclusion In summary, we were able to distinguish between language networks and the random ones by means of their entropy. Language networks differ much from the ER networks and from RMDNs, but are closer to BA and WS networks according to their entropy values. Furthermore, the MDNs from natural languages can be distinguished from random ones by means of their topological characteristics. This finding encourages the use of network approaches in typological studies. That is, constructing an MDN of a language allows us to examine its morphological properties that can be learned from the network topology.

344


Further, we studied some information functionals, each of them seems to highlight a different aspect of the graph, either the distribution of j-spheres, or of the shortest distances between vertices, or of the distance sums in the graph. The entropy based on these functionals allows a perfect distinction of natural language networks from RMDN as well as from random graphs (ER, BA, WS). Information functionals based on distributions of topological properties turned out to be better discriminators than those that are based on properties of single vertices (e.g., f D V vs. f DS ). Surprisingly, good results could be obtained using the functional f./ . Its discriminative potential could raise the F-Measure value from initially 0.857 to 1, which is a remarkable improvement. In general, more extensive parameter studies on this functional should be performed to examine its potential for typological research. MDNs are an example of morphological networks that capture only one aspect of morphology, namely derivation by means of suffixes. However, these networks contain some information about the organization principles of languages which become apparent from their topology. This was demonstrated by our approach. Extensions of the network model including other kinds of morphemes (identified, e.g., by means of a morpheme-segmentation algorithm) should complete the picture. Future work aims to study more sophisticated network models of morphology and their application in typological research. Acknowledgements We would like to express our gratitude to Alexander Mehler and Kirill Medvedev for fruitful discussions and comments. Our special thanks goes to Matthias Dehmer whose useful hints and recommendations helped to improve this chapter. This work is supported by the Linguisitc Networks project (http://www.linguisticnetworks.net/) funded by the German Federal Ministry of Education and Research (BMBF), and by the German Research Foundation Deutsche Forschungsgemeinschaft (DFG) in the Collaborative Research Center 673 “Alignment in Communication.”

References 1. Altmann, G., Lehfeldt, W.: Allgemeine Sprachtypologie. Wilhelm Fink, Germany (1973) 2. Aronoff, M.: Word Formation in Generative Grammar. MIT, Cambridge (1976) 3. Baayen, H.: Quantitative Aspects of Morphological Productivity. In: Geert Booij, J.M. (ed.) Yearbook of Morphology, pp. 109–149. Kluwer, Dordrecht, Boston, London (1991) 4. Baayen, H.: On frequency, transparency, and productivity. Yearbook of Morphology 1992, pp. 181–208 (1992) 5. Barabási, A.L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999) 6. Bauer, L.: Morphological Productivity. Cambridge University Press, Cambridge (2001) 7. Bertinetto, P.M., Noccetti, S.: Prolegomena to ATAM acquisition. Theoretical premises and corpus labeling. Quaderni del Laboratorio di Linguistica della SNS n.6 ns. (2006) 8. Bonchev, D., Rouvray, D.H.: Complexity in Chemistry, Biology, and Ecology. Mathematical and Computational Chemistry. Springer, New York (2005) 9. Brandes, U.: A faster algorithm for betweenness centrality. J. Math. Sociol. 25, 163–177 (2001) 10. Bybee, J.L.: Morphology as Lexical Organization, Chap. 7, pp. 119–141. Academic, London (1988)


345

11. Clahsen, H., Sonnenstuhl, I., Blevins, J.P.: Derivational morphology in the german mental lexicon: a dual mechanism account. In: Baayen, H., Schreuder, R. (eds.), Morphological Structure in Language Processing, Mouton de Gruyter, pp. 125–155, 2006 (2003) 12. Dehmer, M.: Information processing in complex networks: Graph entropy and information functionals. Appl. Math. Comput. 201, 82–94 (2008) 13. Dehmer, M., Varmuza, K., Borgert, S., Emmert-Streib, F.: On entropy-based molecular descriptors: statistical analysis of real and synthetic chemical structures. J. Chem. Inform. Model. 49(7), 1655–1663 (2009) 14. Dressler, W.U., Karpf, A.: The theoretical relevance of pre- and protomorpholgy in language acquisition. Yearbook of Morphology 1994, pp. 99–122 (1995) 15. Erd˝os, P., Rényi, A.: On random graphs. Publicationes Mathematicae 6, 290–297 (1959) 16. Evert, S., Lüdeling, A.: Measuring Morphological Productivity: Is AutomaticPreprocessing Sufficient? In: Rayson, P., Wilson, A., McEnery, T., Hardie, A., Khoja, S. (eds.) Proceedings of the Corpus Linguistics 2001 conference, pp. 167–175. Lancaster (2001) 17. Ferrer i Cancho, R., Mehler, A., Pustylnikov, O., D´ıaz-Guilera, A.: Correlations in the organization of large-scale syntactic dependency networks. In: TextGraphs-2: Graph-Based Algorithms for Natural Language Processing, pp. 65–72 (2007) 18. Ferrer i Cancho, R., Solé, R.V., Köhler, R.: Patterns in syntactic dependency networks. Phys. Rev. E 69, 051, 915 (2004) 19. Freeman, L.C.: Centrality in social networks conceptual clarification. Soc. Network. 1(3), 215–239 (1978-1979) 20. Habermann, M.: Verbale Wortbildung um 1500. Eine historisch-synchrone Untersuchung anhand von Texten Albrecht Dürers, Heinrich Deichlers und Veit Dietrichs. de Gruyter, Berlin (1994) 21. Hotho, A., Nürnberger, A., Paaß, G.: A Brief Survey of Text Mining. J. Lang. Technol. Comput. Ling. (JLCL) 20(1), 19–62 (2005) 22. Köhler, R.: Zur linguistischen Synergetik: Struktur und Dynamik der Lexik. Brockmeyer, Bochum (1986) 23. Konstantinova, E.V.: On some applications of information indices in chemical graph theory. In: General Theory of Information Transfer and Combinatorics. Springer, New York (2006) 24. Konstantinova, E.V., Paleev, A.A.: Sensitivity of topological indices of polycyclic graphs (Russian). Vichisl. Systemy 136, 38–48 (1990) 25. Liu, H.: The complexity of chinese syntactic dependency networks. Phys. A 387, 3048–3058 (2008) 26. Mehler, A.: Structural similarities of complex networks: A computational model by example of wiki graphs. Appl. Artif. Intell. 22, 619–683 (2008) 27. Mehler, A.: A quantitative graph model of social ontologies by example of Wikipedia. In: Dehmer, M., Emmert-Streib, F., Mehler, A. (eds.) Towards an Information Theory of Complex Networks: Statistical Methods and Applications. Birkhäuser, Boston/Basel (2011) 28. Mehler, A., Pustylnikov, O., Diewald, N.: Geography of social ontologies: Testing a variant of the Sapir-Whorf hypothesis in the context of Wikipedia. Comput. Speech Lang. 25(3), 716– 740 (2011) 29. Mehler, A., Lücking, A., Weiß, P.: A network model of interpersonal alignment. Entropy 12(6), 1440–1483 (2010) 30. Plag, I.: Morphological Productivity. Structural Constraints in English Derivation. Mouton de Gruyter, Berlin/New York (1999) 31. Prell, H.P.: Die Ableitung von Verben aus Substantiven in biblischen und nichtbiblischen Texten des Frühneuhochdeutschen. Lang, Frankfurt am Main (1991) 32. Pustylnikov, O.: Modeling learning of derivation morphology in a multi-agent simulation. In: Proceedings of IEEE Africon 2009. IEEE (2009) 33. Abramov, O., Mehler, A.: Automatic Language Classification by Means of Syntactic Dependency Networks. Journal of Quantitative Linguistics (2011) 34. Pustylnikov, O., Schneider-Wiejowski, K.: Measuring morphological productivity. Studies in Quantitative Linguistics 5: Issues in Quantitative Linguistics, pp. 106–125 (2009)

346


35. Schneider-Wiejowski, K.: Sprachwandel anhand von Produktivitätsverschiebungen in der schweizerdeutschen Derivationsmorphologie. Linguistik online 38 (2009) 36. Schultink, H.: Produktiviteit als Morfologisch Fenomeen. Forum der Letteren 2, 110–125 (1961) 37. Shannon, C.E., Weaver, W.: The Mathematical Theory of Communication. University of Illinois Press, Urbana, IL, USA (1997) 38. Snijders, T.A.B.: The degree variance: An index of graph heterogeneity. Soc. Network. 3(3), 163–174 (1981) 39. Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small-world’ networks. Nature 393, 440–442 (1998)

Chapter 12

Information Theory-Based Measurement of Software Edward B. Allen

Abstract Abstractions of software may take the form of complex graphs. This chapter presents methods for measurement of hypergraph abstractions of software, using information theory, rather than counting. Even though the context of this chapter is software engineering, the measurement methods are applicable to any hypergraph. The software-metrics literature often assumes that complex software has more bugs than simple software. Thus, if one measures a software system, one can identify which modules are more likely to have bugs. This chapter presents measures of size, complexity, and coupling in terms of the amount of information, building on formal definitions of these software-metric families proposed by Briand, Morasca, and Basili. Kolmogorov complexity and its information theory-based approximations are the foundation for our measure of size. Excess entropy is the foundation for our measures of complexity and coupling. These concepts make the metrics sensitive to the configuration of hypergraph connections, and thus, may be preferred to corresponding counting metrics. Keywords Information • Software metrics

theory

•

Measurement

•

Software

engineering

MSC2000 Primary 68N30; Secondary 94A17, 68Q30.

E.B. Allen () Department of Computer Science and Engineering, Mississippi State University, Box 9637, Mississippi State, MS 39762, USA e-mail: [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 12, © Springer Science+Business Media, LLC 2011

347

348

E.B. Allen

1 Introduction The goal of this chapter is to introduce applications of information theory to measurement of complex graphs (networks) that represent abstractions of software. Our target audience is scientists and statisticians outside the field of software engineering. Researchers in other fields are well aware of applications of information theory to characterization of complex graphs [14]. A wide variety of entropy-based measures of graph complexity have been proposed in the literature with applications to biology, chemistry, psychology, mathematics, etc. We present applications in software engineering. Even though the context of this chapter is software engineering, we hope that similarly structured problems in other fields may benefit from the methods and rationale presented here. Software products have bugs. This is because software development is a human, labor-intensive, mistake-prone process. Human errors result in defects in the artifacts that software engineers use during development. Defects in intermediate artifacts can result in faults in the delivered product, namely, bugs. When faulty code is executed, the software may fail. Consequently, preventing bugs, finding bugs, and finding them early are high priorities for software engineers. If we could just identify where bugs are most likely, we could find them more efficiently. A “software-quality model” is typically a statistical model where a sample is a module in a large system, the dependent variable is a measure of software quality, such as the number of bugs, and the independent variables are measures of software attributes, such as size (e.g., lines of code) or complexity (e.g., the number of IF and WHILE statements). Given measurements taken early in the software development life cycle, the statistical model can predict the quality of each module, which is directly measurable only later. This chapter focuses on the independent variables, the software metrics. Modeling issues are not considered here. A common thread in the software-metrics literature is the assumption that complex software has more bugs than simple software. However, the word “complexity” is vague, and subject to many definitions. Consequently, a multitude of competing software “complexity” metrics have been proposed in the literature [38]. A traditional software metric [17] typically counts instances of an attribute of an abstraction of software. Counting implies each instance is given equal weight, and the working assumption is that they all have an equal impact on the software-engineering process. In contrast, we adopt information theory instead of counting as our measurement method. This chapter presents some general observations on measurement of software using information theory, and summarizes results presented by Allen et al. [3]. Following usage in the software-engineering community, we use the nouns “measure” and “metric” as synonyms, meaning a method for making a measurement. Khoshgoftaar and Allen’s survey [19] and the follow-up survey of related work by Allen et al. [3] indicate that many information theory-based software metrics are motivated by the idea that the more information that a software product contains, the more complex it appears to a software engineer, and hence, the more difficult to

12 Information Theory-Based Measurement of Software

349

understand [1]. As with traditional software metrics, one then assumes that difficulty in comprehension will lead to more bugs or other indicators of poor software quality. Most of the proposed information theory-based metrics in the literature use entropy as the underlying measurement idea. The remainder of this chapter shows how Kolmogorov complexity is a foundational concept for information theory-based metrics, rather than entropy, discusses graph abstractions of software and their role in software measurement, summarizes formal properties of software metrics, defines our proposed information theorybased software metrics, and discusses implications for further research.

2 Information Theoretic Foundations Kolmogorov complexity is a field of computer science [23] based on a theory defining the information content of finite strings of characters [11, 22, 32]. Informally, the Kolmogorov complexity, K, of a string of characters is the size of the smallest Turing machine program that outputs that string and then terminates. The presentation in this section follows that of Allen et al. [3]. If errors by software engineers are related to the amount of information that they must comprehend, then Kolmogorov complexity is attractive for objectively quantifying the information in a software abstraction. However, there is a practical problem. The function K is not partial recursive, that is, it is not computable exactly [23]. Therefore, we must be satisfied with an information-theoretic approximation. Kolmogorov complexity and information theory are intimately related [12, 22]. Let x be a discrete random variable. Let nx be the number of distinct items in the domain of x, and let pl be the probability of selecting an item l from that domain. The entropy of the distribution of x is calculated by the following [31]: H.x/ D

nx X

pl . log pl /;

(12.1)

lD1

where the base of the logarithm is the number of symbols in the alphabet used to encode the domain of x (e.g., two for binary encoding). We do not assume all items in the domain are equally probable. An instantaneous code is a set of strings (code words) where no string in the set is a prefix of another. Let each code word corresponds to an item in the domain of x. Let E.Len.x// be the expected length per item of an instantaneous code for the domain of x. Information theory gives the following result [13, p. 86]: E.Len.x// H.x/:

(12.2)

In other words, the entropy of the distribution of x is the minimum expected length of an instantaneous code for one sample item.

350

E.B. Allen

If we use an instantaneous code to describe the domain of x, and if a set X consists of n independent identically distributed items from the domain of x, then the minimum expected length of a description of X is n H.x/. Based on the length of Shannon-Fano encoding [13, p.170], this is an approximation to the Kolmogorov complexity of the set of objects X . b / D n H.x/: K.X

(12.3)

In this context, the base of the logarithm in (12.1) is the number of symbols in the code alphabet. Therefore, we interpret n H.x/ as an approximation of the total information in the set X . Suppose one takes a set of n objects, X , identifies an attribute of interest, and partitions the set according to the k distinct values of that attribute [19]. If the objects are independent and identically distributed, a discrete probability distribution can be estimated from the proportion of objects in each cell of the partition. The estimated entropy of such a k-partition is given by the following: b .X / D H

k X nl lD1

n

log

nl ; n

(12.4)

where nl =n is the proportion of objects in the lth cell. A particular case of interest is when our set of data is arranged into an object-predicate table [33], where each row represents an object, and each column represents a predicate of the form, “Does this object have such and such property?” encoded by ones and zeros. Each row i is a pattern of ones and zeros, indexed by l. The row patterns partition the set of objects (rows) and are then suitable for estimating a probability distribution, and the associated entropy of a partition is given by (12.4). By (12.3), an approximation of the Kolmogorov complexity of the object-predicate table, X , is given by the following: b /D K.X

k X lD1

nl nl log : n

(12.5)

3 Abstractions of Software A software product is the manifestation of a multitude of decisions accumulated throughout design and implementation. Because it is infeasible to measure the myriad of design decisions directly, software measurement is always applied to an abstraction of the software. As development progresses, software engineers typically create design artifacts that are themselves abstractions of the intended product. Such software design artifacts often have the form of various kinds of graphs and networks. Consequently, many traditional software metrics are based on graph abstractions of software.

12 Information Theory-Based Measurement of Software Fig. 12.1 Example hypergraph abstraction of software [3]

351 m1 1 1

m2

m3 2

6

5 4

3 2 4

8

7 7

5 6

9

3 m4 10 8 11

12 10

9 13

14

Ordinary graphs consist of sets of nodes, connected by edges where each edge has two end points. In contrast, hypergraphs consist of sets of nodes, connected by hyperedges, where each hyperedge is connected to a subset of the nodes. Figure 12.1 shows a hypothetical hypergraph that could represent four software classes, labeled modules m1 through m4 , containing 14 methods, represented by nodes, using ten public variables, represented by hyperedges. Some public variables are used by only two methods, and thus are connected by an ordinary edge, for example, edge 4. Other public variables are used by more than two methods, such as hyperedges 1 and 2. Method 14 does not use any public variables, and therefore is not connected in this abstraction. We adopt this more general abstraction, because some attributes of software are better represented by hyperedges than ordinary edges [30]. Note that an ordinary graph is a special case of a hypergraph. Also a bipartite ordinary graph can be recast as a hypergraph, where the hyperedges represent the nodes of one side of the bipartite graph. An undirected hypergraph is adequate for our purposes. In this chapter, we use the term “graph” to mean a hypergraph unless otherwise indicated. Formally, we define an abstract “system” and “module” as the following. Definition 1 (System and Module). A system, S, is an abstraction of a software system represented by a graph with n > 0 nodes and with ne 0 hyperedges connecting zero or more of the nodes. A subset of nodes and their incident hyperedges may be designated as a module, mk . Definition 2 (Modular System [9]). A modular system, MS, is a special case of a system, S, whose n nodes are partitioned into nM modules, mk ; k D 1; : : : ; nM .

352

E.B. Allen

This means every node in a modular system is in a module and no node is in multiple modules. Our information theory-based methods for measuring are valid for any abstraction that is a hypergraph. Even though a well-rounded view of software is the combination of many abstractions, we focus on measuring one abstraction at a time. Naturally, the interpretation of the measurements depends on the meaning of the underlying hypergraph. For example, case studies by Allen et al. [3] measured the relationships between C functions and public (global) variables.

4 Property-Based Measurement of Software Briand et al. [9] propose that common terms denoting software metrics, such as “size” and “complexity,” be defined in terms of sets of properties in a formal framework. In particular, they propose properties defining size, complexity, length, coupling, and cohesion. These five terms are commonly used in the softwareengineering literature, but there is no consensus regarding their formal definitions. Briand, Morasca, and Basili see the property-based measurement approach as a way to arrive at consensus definitions for families of metrics that measure an attribute. The property-driven approach lends itself to definition of a wide variety of metrics, each tailored to some special purpose. Perhaps, the property-driven approach to measurement will be useful in other disciplines as well. Their starting point for measurement is any abstraction of software that is an ordinary directed graph. Morasca and Briand [26] extend this to relations in general. Here, we adapt to hypergraphs their property-driven definitions of size, complexity, and coupling. We leave definition and analysis of length and cohesion to future work. This section follows the presentation by Allen et al. [3]. Traditional measures of software size, such as lines of code, have been shown to be highly correlated to bugs and other indicators of lack of software quality [15]. Consequently, size metrics are of prime importance to software engineers. Table 12.1 lists the properties of size adjusted to apply to hypergraphs. Table 12.2 lists the corollaries derived from the properties. Most of the properties in Table 12.1 are focused on nodes. Consequently, the corollaries in Table 12.2 are also focused on nodes. The number of nodes in a system and the number of nodes in a module fulfill the properties in Table 12.1, and therefore each is a counting-based size metric. Note that the properties do not involve hyperedges at all. Software “complexity” has been an important software-metrics concept since seminal works by Halstead [18] and McCabe [24]. Because, in practice, many “complexity” measures are highly correlated to size (e.g., lines of code), Briand et al. [9] are careful to distinguish between the attributes of complexity and size. Table 12.3 lists the properties of complexity adjusted for hypergraphs. Our properties agree with Briand, Morasca, and Basili, except we require a slightly stronger constraint in Property 4, namely, we require “no nodes in common,” but


353

Table 12.1 Properties of size [9] System properties 1. Nonnegativity. The size of a system is nonnegative. 2. Null value. The size of a system is zero if its set of nodes is empty. 3. Module additivity. Given a system, S, having modules, m1 and m2 , such that every node in S is in m1 or m2 , but not both, the size of this system is equal to the sum of the sizes of the modules m1 and m2 . Size.S/ D Size.m1 jS/ C Size.m2 jS/: Module properties 4. Nonnegativity. The size of a module is nonnegative. 5. Null value. The size of the module is zero if its set of nodes is empty. 6. Monotonicity. Adding a node to a module does not decrease its size.

Table 12.2 Corollaries for size metrics [9] 1. Node additivity. Given a modular system, MS, where each node is a module, mk ; k D 1; : : : ; nM , the size of the modular system is given by: Size.MS/ D

nM X

Size.mk jMS/:

kD1

2. Monotonicity. Adding a node to a system does not decrease its size. 3. General module additivity. Given a system, S, with any two modules, m1 and m2 , such that every node in S is a node in m1 or m2 or both, the size of the system is not greater than the sum of the sizes of the pair of modules. Size.S/ Size.m1 jS/ C Size.m2 jS/: 4. Merging of modules. Given a system, S, with any two modules, m1 and m2 such that every node in S is a node in m1 or m2 or both, construct S0 such that m1 and m2 in S are replaced by m1[2 D m1 [ m2 in S0 . Size.S0 / Size.m1 jS/ C Size.m2 jS/:

Briand, Morasca, and Basili require “no edges in common” in their corresponding property. Briand, Morasca, and Basili’s properties are based on directed graphs. Property 3 asserts that the direction of hyperedge connections is not relevant to complexity. The number of connections in a system and the number of connections in a module fulfill the properties in Table 12.3, and thus are examples of countingbased complexity metrics. Properties 6 and 9, Monotonicity, illustrate how the set of properties is focused on hyperedges, rather than on nodes. A standard textbook design rule is, “Minimize coupling and maximize cohesion.” However, there is a wide variety of metrics for these attributes [5, 6]. Here, we consider coupling. Table 12.4 lists the properties of coupling adjusted to apply

354

E.B. Allen

Table 12.3 Properties of complexity [3] System properties 1. Nonnegativity. The complexity of a system is nonnegative. 2. Null value. The complexity of a system is zero if its set of hyperedges is empty. 3. Symmetry. The complexity of a system does not depend on the convention chosen to represent the direction of hyperedge connections. 4. Module monotonicity. Given a System, S, with any two modules, m1 and m2 , that have no nodes in common, the complexity of the system is no less than the sum of the complexities of the two modules. Complexity.S/ Complexity.m1 jS/ C Complexity.m2 jS/: 5. Disjoint module additivity. Given a system, S, composed of two disjoint modules, m1 and m2 , the complexity of the system is equal to the sum of the complexities of the two modules. Complexity.S/ D Complexity.m1 jS/ C Complexity.m2 jS/: 6. Corollary: Monotonicity. Adding a hyperedge to a system does not decrease its complexity. Module properties 7. Nonnegativity. The complexity of a module is nonnegative. 8. Null value. The complexity of a module is zero if its set of intermodule and intramodule hyperedges is empty. 9. Monotonicity. Adding an intermodule or intramodule hyperedge to a module does not decrease its complexity.

to hypergraphs. We make no distinction between inbound and outbound coupling, because an undirected graph does not distinguish direction. Properties 2, 3, 7, and 8 of Table 12.4, say “intermodule” hyperedges, whereas the corresponding properties of Briand et al. [9] say “output” edges. The number of connections to intermodule hyperedges in a system and the number of connections to intermodule hyperedges incident to a module fulfill the properties in Table 12.4, and thus, are examples of counting-based coupling metrics. Similar to complexity, the monotonicity properties (Properties 3 and 8) focus on hyperedges. In the case of coupling, only intermodule hyperedges are of interest.

5 Information Theory-Based Software Metrics Counts of nodes and connections clearly fulfill the specified properties. However, these metrics give equal weight to nodes or connections without consideration of the pattern of connections. Information theory provides a way to give more weight to rare patterns and less weight to the commonplace. The presentation in this section follows that of Allen et al. [3]. We extend the idea of an abstract system, S, to explicitly model the system boundary and the lack of connection to the environment.


355

Table 12.4 Properties of coupling [3] System properties 1. Nonnegativity. Coupling of a modular system is nonnegative. 2. Null value. Coupling of a modular system is zero if its set of intermodule hyperedges is empty. 3. Monotonicity. Adding an intermodule hyperedge to a modular system does not decrease its coupling. 4. Merging of modules. If two modules, m1 and m2 , are merged to form a new module, m1[2 , that replaces m1 and m2 , then the coupling of the modular system with m1[2 is not greater than the coupling of the modular system with m1 and m2 . 5. Disjoint module additivity. If two modules, m1 and m2 , which have no intermodule hyperedges between nodes in m1 and nodes in m2 , are merged to form a new module, m1[2 , that replaces m1 and m2 , then the coupling of the modular system with m1[2 is equal to the coupling of the modular system with m1 and m2 . Module properties 6. Nonnegativity. Coupling of a module is nonnegative. 7. Null value. Coupling of a module is zero if its set of intermodule hyperedges is empty. 8. Monotonicity. Adding an intermodule hyperedge to a module does not decrease its module coupling. 9. Merging of modules. If two modules, m1 and m2 , are merged to form a new module, m1[2 , that replaces m1 and m2 , then the module coupling of m1[2 is not greater than the sum of the module coupling of m1 and m2 . 10. Disjoint module additivity. If two modules, m1 and m2 , which have no intermodule hyperedges between nodes in m1 and nodes in m2 , are merged to form a new module, m1[2 , that replaces m1 and m2 , then the module coupling of m1[2 is equal to the sum of the module coupling of m1 and m2 .

Definition 3 (System Graph [4]). The system graph, S , of a system, S, with n nodes is all nodes in S and all its hyperedges, plus a disconnected node modeling the system’s environment. Without loss of generality, index the environment node as i D 0, and the nodes in S as i D 1; : : : ; n. Because our abstractions of software are hypergraphs, our goal is to measure patterns of connections in a system graph. We label each node with the set of hyperedges that are incident to it. Undirected graphs are sufficient for our purposes, and the topology is not restricted (e.g., acyclic graphs are not required). To fully specify an undirected graph, we use a nodes hyperedges table where each cell indicates whether the node is connected to the hyperedge, or not, encoded as one or zero, respectively [4]. Consequently, each node’s label (i.e., the set of incident hyperedges) is encoded as the binary pattern of values in a row of the table. This kind of table is essentially an object-predicate table [33], where nodes are objects (rows) and each predicate (column) is of the form, “Is this node related to other nodes by this hyperedge?” This implies that each node and each hyperedge has an identity, which is the case with abstractions of software. This contrasts with characterizations of complex graphs that are based on topology alone. Dehmer [14] summarizes such contributions.

356

E.B. Allen

Suppose we have a system, S. We model the designer’s preferences for connecting nodes to hyperedges as a probability distribution. As a simplifying assumption, our analysis excludes from measurement the design decisions of how many nodes and how many hyperedges the graph includes. Hence, we model the number of nodes, n, in S and the number of hyperedges, ne , as given. Specifically, we model the system graph S as a set of statistically independent identically distributed samples from a probability distribution on the possible row patterns of its nodes hyperedges table, pl ; l D 1; : : : ; nS , where nS is the number of possible distinct row patterns. The entropy of the distribution of row patterns is the following: nS X pl . log pl /: (12.6) H.S / D lD1

In this application, entropy is the average information per node. We use base two logarithms in information theoretic calculations. Consequently, the unit of measure is a bit. The number of rows in S , including the environment, is n C 1. Let nl be the number of rows with pattern l. The proportion of rows with pattern l is an estimate of the probability of pattern l [33], namely, pOl D nl =.n C 1/. By (12.6), entropy can be estimated by the following: nS X nl nl b log ; (12.7) H .S / D nC1 nC1 lD1

b .S / D H

n X i D0

1 log pOL.i / ; nC1

(12.8)

where L.i / is a function that gives the label l (i.e., row pattern) of node i . Note that the summation in (12.8) is over the set of nodes (i ), rather than the set of distinct row patterns (l), as in (12.6). Because of the way the probabilities of patterns are estimated based on row patterns, our information theory-based metrics are not affected when redundant columns are added to an object-predicate table. This means that redundant hyperedges do not affect the information theory-based measurements.

5.1 An Information Theory-Based Size Metric Without loss of generality, if we use an instantaneous code to describe row patterns, then the minimum expected length of a description [13] of a set S consisting of n C 1 nodes is .n C 1/ H.S /. b / D .n C 1/ H b .S / D K.S

n X log pOL.i / : i D0

(12.9)


357

Thus, each node contributes to the minimum expected length of a description of S . Because we estimate probabilities by proportions pOl D nl =.n C 1/, the existence of the environment node assures that the estimated probability of a disconnected node is nonzero, pOL.0/ 1=.n C 1/ > 0. This facilitates monotonicity properties of metrics. We interpret the minimum expected length of a description of S as the amount of information in S , which is an approximation of the Kolmogorov complexity. Definition 4 (Information Size of a System [2]). The size of a system, S, is the amount of information in its system graph, S , less the contribution of the environment. Size.S/ D .n C 1/ H.S / log pL.0/ : (12.10) The estimated information contribution of the environment node is log pOL.0/ . Consequently, by (12.9), the size of a system is estimated by the following: Estimated Size.S/ D

n X

log pOL.i / :

(12.11)

i D1

The summation is over the set of nodes, beginning with i D 1 instead of i D 0. Therefore, we can allocate the total information to constituent modules. Definition 5 (Information Size of a Module [2]). The size of a module, mk , in a system, S, is the information in its system graph contributed by the module. Size.mk jS/ D

X log pL.i / :

(12.12)

i 2mk

This definition conforms to Corollary 1 in Table 12.2, namely, the system size is the sum of the module sizes in a modular system. The more that row patterns (i.e., the pattern of incident hyperedges) are repeated, the information in the system is less, and the size is smaller. Thus, this measure of size is sensitive to the pattern of connectivity, but the number of nodes is not. In the extreme case when a graph, S0 , has no connecting hyperedges, the b .S0 / D 0, estimated probability of a row pattern of all zeros is pO0 D 1, and thus, H and consequently, Size.S0 / D 0. If most row patterns are unique, then information module size is highly correlated with the number of nodes [3]. If all row patterns in a module, mk , are unique, then information module size is proportional to the number of rows in that module, nk . Size.mk jS/ D nk log

1 : nC1

(12.13)

358

E.B. Allen

If an ordinary connected graph, S, has three or more nodes, not considering an environment node, then all the row patterns will be unique [3]. Thus, information size has no advantage over the number of nodes as a measure of size when ordinary graphs are under consideration. Our measures of size are members of the family of size metrics defined by the property set. The advantage of using an approximation of Kolmogorov complexity as the basis for measurement, rather than counting, is sensitivity to the patterns of connections. Our working hypothesis is that size measured this way is related to comprehension and hence to bugs. However, this hypothesis has not been empirically investigated yet.

5.2 An Information Theory-Based Complexity Metric Because the properties of complexity listed in Table 12.3 emphasize hyperedges connecting nodes, we define a working abstraction, a hyperedges-only graph, S# , that is a subgraph of the system graph, S, Definition 6 (Hyperedges-Only Graph [2]). Given a system, S, its hyperedgesonly graph, S# , consists of all nodes in S connected to hyperedges and all its hyperedges. The corresponding system graph is denoted by S # . In other words, a hyperedges-only system graph, S # , is similar to its underlying system graph, but all disconnected nodes are removed, except the environment node. Consider the following subgraph of a hyperedges-only graph. Definition 7 (Node Subgraph [4]). Given a hyperedges-only graph, S# , the node subgraph, Si , consists of all the nodes in S# and the hyperedges of S# connected to the i t h node. Its system graph is denoted by Si . A node subgraph, Si , may very well have many disconnected nodes, because all nodes in S # are included, but the set of hyperedges is limited in each Si . Recall that when measuring size, we model S as a probability distribution on its object-predicate table’s row patterns. Similarly, we model each Si as a probability distribution estimated by the proportions of each pattern in its object-predicate table. Given a hyperedges-only graph S# with n nodes, the following is similar to (12.8): b .Si / D H

n X

1 log pOLi .j / ; nC1 j D0

(12.14)

where Li .j / is a function that gives the pattern index, l, of the j th row of Si . Each row of each Si is a subset of the corresponding row of S # , and conversely, # S represents the joint distribution of all the Si . Consequently, we can apply a result of information theory [13, p.28]. n X i D0

H.Si / H.S # /:

(12.15)


359

Watanabe [35] shows that the difference between the sides of this equation is a measure of the relationships among the components. Excess entropy [33] of S # is defined as the following. C.S # / D

n X

H.Si / H.S # /:

(12.16)

i D0

Connected nodes are related to each other by the presence of a hyperedge. If the Si are highly related to each other by common hyperedge connections and common disconnected nodes, then the excess entropy is high. If the hyperedges have small interaction with each other, then excess entropy is low [33]. We extend Mohanty’s idea [25] of measuring software interactions using excess entropy. Since a node is a sample from the probability distribution, we multiply excess entropy by the number of nodes, .n C 1/ C.S # /, yielding total information in relationships. Definition 8 (Information Complexity of a System [2]). The complexity of a system, S, is given by the amount of information in relationships in its hyperedgesonly graph, less the contribution of the environment. Complexity.S/ D

n X

Size.S#i / Size.S# /:

(12.17)

i D1

When probabilities refer to S # and its Si , (12.17) is derived from (12.8), (12.11), (12.14), and (12.16). Note the following: 0 Estimated Complexity.S/ D @

n n X X

1 n X log pOLi .j / A log pOL.i / (12.18)

i D1 j D1

i D1

b.S / Estimated Complexity.S/ D .n C 1/C #

n X

log pOLi .0/ log pOL.0/

! :

i D1

(12.19) If hyperedges are related to each other through the pattern of connections to nodes, then Complexity.S/ is high. The estimated information complexity of a system can be allocated to its constituent modules. Definition 9 (Information Complexity of a Module [2]). The complexity of a module, mk , in a system, S, is its contribution to the complexity of the system, given by: X Complexity.mk jS/ D Size.S#i / Size.mk jS# /: (12.20) i 2mk

360

E.B. Allen

The information complexity of a modular system, MS, is the sum of the information complexities of its modules, because the modules do not overlap. Although our goal was to conform to the properties defined by Briand et al. [9], this definition of information module complexity, Complexity.mk jS/, does not conform to Property 7 regarding module nonnegativity in Table 12.3 for certain extreme cases. If a module consists of a single node that is connected by one hyperedge to all other nodes, then its information module complexity will be negative (except in the case of a fully connected graph). It appears that this case is of little practical importance in software engineering. Our measures of complexity are members of the family of complexity metrics defined by the property set, with the exception of the nonnegativity property of module complexity in extreme cases. The advantage of using a function of excess entropy, rather than counting, as the basis for measurement is sensitivity to the pattern of hyperedges. Like information size, we hypothesize that complexity measured this way is related to comprehension and hence to bugs. However, this hypothesis has not been empirically investigated with human subjects yet.

5.3 An Information Theory-Based Coupling Metric Because the properties of coupling in Table 12.4 emphasize hyperedges connecting nodes in different modules in a modular system, MS, we define a working abstraction, an intermodule-hyperedges graph, MS , that is a subgraph of the modular system graph, MS. Definition 10 (Intermodule-Hyperedges Graph [4]). Given a modular system, MS, its intermodule-hyperedges graph, MS , consists of all nodes in MS and all its intermodule hyperedges. The corresponding system graph is denoted by S . Our definition of coupling of a modular system builds on our definition of complexity [4]. Both are based on configurations of hyperedges, but the set of coupling hyperedges is different. Definition 11 (Information Coupling of a Modular System). The coupling of a modular system, MS, is the amount of information in intermodule relationships in its system graph, less the contribution of the environment. Coupling.MS/ D Complexity.MS /:

(12.21)

Like complexity, system-level information coupling can be allocated to modules. Definition 12 (Information Coupling of a Module). The coupling of a module, mk , in a modular system, MS, is its contribution to the coupling of the system, given by: Coupling.mk jMS/ D Complexity.mk jMS /:

(12.22)


361

The underlying graphs of Complexity.S/ and Coupling.S/ are different, but the measurement method is the same, namely, a function of excess entropy. Our measures of coupling are similar to our measures of complexity. The advantage of using a function of excess entropy for coupling, rather than counting, is sensitivity to the pattern of intermodule hyperedges.

6 Discussion For a software metric to be accepted by the software-engineering community, it should be theoretically valid from measurement-theory perspective [7], and it should be empirically shown to be valuable in the context of software engineering. Our proposed metrics fulfill the following theoretical validation criteria proposed by Kitchenham et al. [20]: • Measurements are allowed to take different values, so that measurements can usefully distinguish systems. • Multiple systems are allowed to have the same values, so that a measurement is not merely an identifier. • Each metric is a valid representation of its attribute, because it essentially conforms to the properties of Briand et al. [9], with the exception of module complexity, and the similar module coupling in extreme cases. The method of measurement must conform to the intuitive meaning of the metric’s name. • Each metric has an accepted unit of measure, namely, a bit, which is a widely used unit of measure for information. • All the metrics are compatible with the ratio scale-type [7]. The ratio scale-type is preferred in science and engineering, because ratio-scale measurements are more easily incorporated into mathematical models. • Given a hypergraph, the measurement procedures are well defined. Thus, the information theory-based metrics have the theoretical properties that one would expect of a scientific measure. The goal of empirical validation is to demonstrate the potential usefulness of a software metric to practicing software engineers. This is often done through case studies, where a body of data from a software development project is the basis for a hypothetical scenario. For example, one might demonstrate that a statistical model, where software metrics are the independent variables, yields useful predictions of software quality measurements, such as the number of bugs in each module [29]. Case studies provide evidence of potential usefulness, but not conclusive proof. Empirical validation of information theory-based software metrics has been limited [3, 19]. Allen et al. [3] analyzed three sets of data from various size systems to illustrate how the metrics behave. Further empirical validation studies are certainly needed, if practicing software engineers are to adopt information theory-based metrics into routine use.

362

E.B. Allen

7 Conclusions This chapter presents a summary of information theory-based software metrics for size, complexity, and coupling, and their rationale. Similar measurement methods for software length and cohesion remain for future work. Families of such metrics are formally defined by property sets, and the proposed metrics are members of their respective families. The property-based approach to measurement may be applicable to other fields where vague terms are often used to describe measured attributes. All software measurements are based on abstractions of software and not the software itself. This work is based on hypergraph abstractions of software. The interpretation of a metric depends on what the hypergraph represents in the real world. The measurement methods are applicable to any hypergraph, so they may find application in other fields where a hypergraph is a useful abstraction. Kolmogorov complexity and its information theory-based approximations are the foundation for our measure of size. Excess entropy is the foundation for our measures of complexity and coupling. These concepts are sensitive to the configuration of hypergraph connections, and thus may be preferred to corresponding counting metrics. These measurement methods may be useful in other fields of study where graph connections are important. The software-metrics literature rarely draws on the theories and experiments of cognitive science, except for casual motivation. Similarly, the literature on information theory-based software metrics has yet to make theoretical and empirical connections to cognitive science. In the future, such cross-disciplinary research has the potential to significantly enrich the foundations of information theory-based software metrics. Acknowledgements This work was supported in part by grant CCR-0098024 from the National Science Foundation. The findings and opinions in this chapter belong solely to the author and are not necessarily those of the sponsor. We thank Sampath Gottipati, Rajiv Govindarajan, Taghi M. Khoshgoftaar, and Ye Chen, who were coauthors of related work with us. We thank the Empirical Software Engineering research group at Mississippi State University for helpful discussions.

References 1. Allen, E.B.: Information theory and software measurement. PhD thesis, Florida Atlantic University, Boca Raton, Florida, August 1995. Advised by Taghi M. Khoshgoftaar 2. Allen, E.B.: Measuring graph abstractions of software: An information-theory approach. In Proceedings: Eighth IEEE symposium on software metrics, pages 182–193, Ottawa, Canada, June 2002. IEEE Computer Society 3. Allen, E.B., Gottipati, S., Govindarajan, R.: Measuring size, complexity, and coupling of hypergraph abstractions of software: An information-theory approach. Software Qual. J. 15(2), 179–212 (2007)


363

4. Allen, E.B., Khoshgoftaar, T.M.: Measuring coupling and cohesion: An information-theory approach. In Proceedings of the sixth international software metrics symposium, pages 119–127, Boca Raton, Florida, November 1999. IEEE Computer Society 5. Briand, L.C., Daly, J.W., Wüst, J.: A unified framework for cohesion measurement in objectoriented systems. Empir. Software Eng. Int. J. 3(1), 65–117 (1998) 6. Briand, L.C., Daly, J.W., Wüst, J.K.: A unified framework for coupling measurement in objectoriented systems. IEEE Trans. Software Eng. 25(1), 91–121 (1999) 7. Briand, L.C., El Emam, K., Morasca, S.: On the application of measurement theory in software engineering. Empir. Software Eng. Int. J. 1(1), 61–88 (1996). See [8, 37] 8. Briand, L.C., El Emam, K., Morasca, S.: Reply to “Comments to the paper: Briand, El Emam, Morasca: On the application of measurement theory in software engineering”. Empir. Software Eng. Int. J. 2(3), 317–322 (1997). See [7, 37] 9. Briand, L.C., Morasca, S., Basili, V.R.: Property-based software engineering measurement. IEEE Trans. Software Eng. 22(1), 68–85 (1996). See comments in [10, 28, 39] 10. Briand, L.C., Morasca, S., Basili, V.R.: Response to: Comments on “Property-based software engineering measurement”: Refining the additivity properties. IEEE Trans. Software Eng. 23(3), 196–197 (1997). See [9, 28] 11. Chaitin, G.J.: On the length of programs for computing finite binary sequences. J. ACM 13(4), 547–569 (1966) 12. Chaitin, G.J.: A theory of program size formally identical to information theory. J. ACM 22(3), 329–340 (1975) 13. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991) 14. Dehmer, M.: Information-theoretic concepts for the analysis of complex networks. Appl. Artif. Intell. 22(7), 684–706 (2008) 15. El Emam, K., Benlarbi, S., Goel, N., Rai, S.N.: The confounding effect of class size on the validity of object-oriented metrics. IEEE Trans. Software Eng. 27(7), 630–650 (2001). See [16] 16. Evanco, W.M.: Comments on “The confounding effect of class size on the validity of objectoriented metrics”. IEEE Trans. Software Eng. 29(7), 670–672 (2003). See [15] 17. Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach, 2nd edn. PWS Publishing, London (1997) 18. Halstead, M.H.: Elements of Software Science. Elsevier, New York (1977) 19. Khoshgoftaar, T.M., Allen, E.B.: Applications of information theory to software engineering measurement. Software Qual. J. 3(2), 79–103 (1994) 20. Kitchenham, B.A., Pfleeger, S.L., Fenton, N.E.: Towards a framework for software measurement validation. IEEE Trans. Software Eng. 21(12), 929–944 (1995). See comments in [21,27] 21. Kitchenham, B.A., Pfleeger, S.L., Fenton, N.E.: Reply to: Comments on “Towards a framework for software measurement validation”. IEEE Trans. Software Eng. 23(3), 189 (1997). See [20, 27, 36] 22. Kolmogorov, A.N.: Logical basis for information theory and probability theory. IEEE Trans. Inform. Theory 14(5), 662–664 (1968) 23. Li, M., Vitányi, P.M.B.: Two decades of applied Kolmogorov complexity. In Proceedings of the third annual structure in complexity theory conference, pages 80–101, Washington, DC, June 1988 24. McCabe, T.J.: A complexity measure. IEEE Trans. Software Eng. 2(4), 308–320 (1976) 25. Mohanty, S.N.: Entropy metrics for software design evaluation. J. Syst. Software 2, 39–46 (1981) 26. Morasca, S., Briand, L.C.: Towards a theoretical framework for measuring software attributes. In Proceedings of the Fourth International Symposium on Software Metrics, pages 119–126, Albuquerque, New Mexico, November 1997. IEEE Computer Society 27. Morasca, S., Briand, L.C., Basili, V.R., Weyuker, E.J., Zelkowitz, M.V.: Comments on “Towards a framework for software measurement validation”. IEEE Trans. Software Eng. 23(3), 187–188 (1997). See [20, 36] 28. Poels, G., Dedene, G.: Comments on “Property-based software engineering measurement”: Refining the additivity properties. IEEE Trans. Software Eng. 23(3), 190–195 (1997). See [9]

364

E.B. Allen

29. Schneidewind, N.F.: Methodology for validating software metrics. IEEE Trans. Software Eng. 18(5), 410–422 (1992) 30. Schütt, D.: On a hypergraph oriented measure for applied computer science. In Digest of Papers: COMPCON 77 Fall, pages 295–296, Washington, DC, September 1977. IEEE Computer Society. Abstract only 31. Shannon, C.E., Weaver, W.: The Mathematical Theory of Communication. University of Illinois Press, Urbana, Illinois (1949) 32. Solomonoff, R.J.: A formal theory of inductive inference, part 1 and part 2. Inform. Contr. 7, 1–22, 224–254 (1964) 33. van Emden, M.H.: Hierarchical decomposition of complexity. Machine Intelligence 5, 361–380 (1970). See also [34] for details 34. van Emden, M.H.: An Analysis of Complexity. Number 35 in Mathematical Centre Tracts. Mathematisch Centrum, Amsterdam (1971) 35. Watanabe, S.: Information theoretical analysis of multivariate correlation. IBM J. Res. Dev. 4(1), 66–82 (1960) 36. Weyuker, E.J.: Evaluating software complexity measures. IEEE Trans. Software Eng. 14(9), 1357–1365 (1988) 37. Zuse, H.: Comments to the paper: Briand, Emam, Morasca: On the application of measurement theory in software engineering. Empir. Software Eng. Int. J. 2(3), 313–316 (1997). See [7, 8] 38. Zuse, H.: A Framework for Software Measurement. Walter de Gruyter and Co., Berlin (1997) 39. Zuse, H.: Reply to: “Property-based software engineering measurement”. IEEE Trans. Software Eng. 23(8), 533 (1997). See [9]

Chapter 13

Fair and Biased Random Walks on Undirected Graphs and Related Entropies Philippe Blanchard and Dimitri Volchenkov

Abstract The entropy rates of Markov chains (random walks) defined on connected undirected graphs are well studied in many surveys. We study the entropy rates related to the first-passage time probability distributions of fair random walks, their relative (Kullback–Leibler) entropies, and the entropy related to two biased random walks – with the random absorption of walkers and the shortest paths random walks. We show that uncertainty of first-passage times quantified by the entropy rates characterizes the connectedness of the graph. The relative entropy derived for the biased random walks estimates the level of uncertainty between connectivity and connectedness – the local and global properties of nodes in the graph. Keywords Entropy of graphs • First-passage times • Random walks on graphs MSC2000 Primary 82B41; Secondary 28D20, 68R10.

1 Introduction Most of the graphs we have to deal with are rather complex to be analyzed visually, but typically are not large enough to be investigated by statistical methods of complex networks theory. Probably, the main difficulty we experience while investigating a graph structure comes from graph automorphisms. Indeed, any graph can be drawn in many different ways, and we may never be sure whether two homologous graphs are equivalent, as they look different. Random walks may help

D. Volchenkov () The Center of Excellence Cognitive Interaction Technology (CITEC), University of Bielefeld, Postfach 100131, 33501 Bielefeld, Germany e-mail: [email protected]; [email protected] M. Dehmer et al. (eds.), Towards an Information Theory of Complex Networks: Statistical Methods and Applications, DOI 10.1007/978-0-8176-4904-3 13, © Springer Science+Business Media, LLC 2011

365

366

P. Blanchard and D. Volchenkov

us by mapping all graph automorphisms into one and the same set of properties, since their statistical properties are independent of graph symmetries [4, 5]. In the simplest algorithm thoroughly studied in literature [28], a walker entering a graph vertex at time t chooses its next location at time t C 1 randomly, with equal probability, among all available neighboring vertices – we call it as the “fair” random walks. Then, the recurrence time to a node indicates how long a random walker must wait to revisit the site. It is known that for a stationary, discretevalued stochastic process, the expected recurrence time to return to a state is just the reciprocal of the probability of this state [25]. The first-passage time to a node is another important characteristic the node acquires with respect to random walks – it is the expected number of steps required for a random walker to visit the node for the first time starting the walk from the arbitrary node of the graph. Intuitively, we may think that both characteristic times of the node should be positively related to each other: the easier was the first access to the site, the more often we can expect to find the walker in that again. Surprisingly, this is not always true. In applications, those graphs for which the return times to the vertices are positively related to the first-passage times to them are sometimes called “intelligible” [4]. Furthermore, in many applications and in the agent-based models defined on graphs in particular, the actual dynamics of agents is supposed to depart from random one in favor of some “rational” strategy. For instance, we may consider a number of agents moving from different sources to various destinations in the graph following the shortest paths between them. It is important to mention that such a rational strategy can also be randomized to a biased random walk, in which the walker prefers to move to those nodes, with the highest values of the betweenness centrality index. While studying different random walks defined of undirected graphs such as the models described above, we always have deal with some probability distributions p on the graph vertex set V , and then the entropy H.p; G/ is the natural graph theoretic functional on the graph G with respect to p. We may also be interested in comparison of the different probability distributions, p and p0 , derived either from two different random walks, or from one and the same random walk but at the different stages of its evolution. A general framework for defining the entropy of a graph has been developed in [14]. The structural information content of graphs as their corresponding graph entropy has been defined in [15] on a ground of on local vertex functionals obtained by calculating j -spheres via the algorithm of Dijkstra. In Sect. 2, we give a brief overview of the different models of random walks defined on undirected graphs and introduce the entropy, and in Sect. 3 we investigate the statistical properties of random walks defined on the spatial graphs of five compact urban patterns and the Petersen graph. In particular, we show that the entropy rate of fair random walks generalizes the notion of space dimension for the graph. Entropy rates introduced for the first-passage times to the graph nodes characterize the entire connectedness of the graph.

13 Fair and Biased Random Walks on Undirected Graphs and Related Entropies

367

2 Random Walks and Entropies 2.1 Entropy as a Physical Dimension of a Graph The stationary distribution of the simplest random walk model described in Sect. 1 defined on a undirected graph G.V; E/ is i D

ki ; 2M

(13.1)

where ki D deg.i / is the number of neighbors the node i 2 V has in G, and jEj D M . The stationary distribution (13.1) is uniform if the graph is regular. Interestingly, it does not depend upon the size jV j D N of the graph G, but only on the total number of edges, M , and the local property of the vertex. Consequently, the recurrence time of random walks to the node i is inversely proportional to i , ri D

2M ki

(13.2)

being dependent upon the local property of the vertex. While in R3 , a walker has three basic directions in which she can move from a particular point – these are physical dimensions of our space. While simulating the diffusion equation uP D 4u

(13.3)

for a scalar function u defined on a regular d -dimensional lattice La D aZd , with the lattice scale length a, one uses the discrete representation of the Laplace operator 4, 3 2 X 1 1 ut C1 .x/ ut .x/ D ut .y/ k ut .x/5 ; 2 4 k a y2U

(13.4)

x

where Ux is the neighborhood of the node x in the lattice La . The degree of each site of the lattice uniformly equals k D 2d

(13.5)

and therefore the parameter d in (13.4) can be naturally interpreted as the physical dimension of space. Being defined on the undirected graph G.V; E/, the discrete Laplace operator (13.4) has the pretty same form, excepting for the cardinality k which now depends upon the site, kx D deg.x/;

368


so that the parameter ıx D log2 kx D log2

2M rx

(13.6)

can be considered as the local analog of the physical dimension d at the node x 2 V [37]. An interesting question arises in concern with (13.6), namely if it is possible to define a universal, global dimension for the graph G that can be considered as generalizing the space dimension in lattices. Below, we show that this can be done on a statistical ground, by estimating the spreading of a set of independent random walkers. In information theory [12], such a spreading is measured by means of the entropy rate, which may be considered as the informational analog of the physical dimension of space. It is clear that the number of possible paths in an undirected graph G.V; E/, jXn j D

X

.An /ij ;

(13.7)

i;j 2V

where An is the nth power of the adjacency matrix of the graph G, grows up exponentially with the path length n. Therefore, the probability to observe a long enough typical random path fX1 D i1 ; : : : Xn D in g 2 X decreases asymptotically exponentially with n 1, 2n .H.X /C"/ Pr ŒfX1 D i1 ; : : : Xn D in g 2n .H.X /"/ ;

(13.8)

where the parameter n H.X / measuring the uncertainty of paths in random walks (entropy) grows asymptotically linearly with n at a rate H.X / D lim

n!1

log2 jXn j ; n

(13.9)

which is called the entropy rate of random walks. In order to proceed further, we should answer the following question: which random path can asymptotically be considered as a typical one? It is well known that if being defined on an undirected non-bipartite graph, the distribution of random walks always tends to , with the mixing rate ˇ ˇ ˇ .t / ˇ ! D lim sup max ˇpij j ˇ ; t !1

.t /

i;j 2V

(13.10)

in which pij is the transition probability between vertices i and j in t > 0 steps in the random walk. Therefore, while interesting in statistics of very long walks, it seems natural to consider those of them satisfying the stationary distribution of walkers as being typical for the given walk.


369

For the Markov chain X , defined in Sect. 1 the one-step transition probability of the random walk between vertices is defined by: Tij D

Aij ki

(13.11)

and therefore the entropy rate (13.8) is given by: H D

X i 2V

i

X j 2V

Tij log2 Tij

D

X ki X Aij Aij ; log2 2M ki ki i 2V

j 2V

(13.12) in which it is assumed that 0 log.0/ D 0: From (13.12), it is clear that H D

X ıi 1 X ki log2 ki D 2M i 2V ri i 2V

D

X

i ıi D hıi i ;

(13.13)

i 2V

where hıi i means the average of the local dimension (13.6) over the stationary distribution of random walks i : From (13.13), it follows that the entropy rate H is nothing else but the averaged local dimension ıi of the graph G. In information theory [12], the entropy rate (13.13) is important as a measure of the average message size required to describe a stationary random walk defined on the graph G. Provided we use the binary code, we need approximately nH bits to describe the typical long enough path of length n. The entropy rates have recently been used in [6, 20] as a measure characterizing topological properties of complex networks.

2.2 First-Passage Time to a Vertex The recurrence time to a vertex is one of characteristic times the vertex has with respect to random walks X . Starting a random walk on the graph from a random initial vertex, we count the number of random steps a walker reach a given vertex i 2 V for the first time. Indeed, this number is random and varies from one trial walk to another. However, we can take an average over very many trial random walks and obtain its expectation value, the first-passage time, .i/ D E f n W X0 D x0 ; X1 D x1 ; : : : Xn D i j x0 ¤ i; x1 ¤ i; : : : Xn1 ¤ i g : (13.14) A classical walker cannot be seen in two different vertices simultaneously, so it seems intuitive that the first-passage times to vertices of the graph depend on each other. This dependence can be described by means of the first-passage time probability distributions. Below, we elaborate this issue in more details.

370


Given a random walk X , we denote the probability of transition between vertices i and j in t > 0 steps by: .t / (13.15) pij D T t ij ; where T t is the t th power of the transition operator (13.15). It is easy to obtain that the limit lim Tt D 1 >

(13.16)

t !1

exists and is a positive stochastic matrix. In (13.16), 1 D .1; 1; : : : ; 1/, and is the vector corresponding to the stationary distribution (13.1). The first-hitting probabilities of that a random walker starting at i 2 V reaches j for the first time precisely in t steps, .t /

qij D Pr Œ vt D j; vl ¤ j; l ¤ 1; : : : ; t 1j v0 D i ;

.0/

qij D 0

(13.17)

.t /

are related to the transition probabilities pij by: .t /

pij D

t X

.s/ .t s/

qij pjj

:

(13.18)

sD0

We are interested in the first-hitting time hij D

X

.t /

t qij ;

(13.19)

t 0

that is the expected number of steps before node j is visited, starting from node i . The first-hitting time hij satisfies the equation [28] hij D 1 C

X

hvj Tvi ;

hi i D 0; 8i 2 V;

(13.20)

i v

reflecting the fact that the first step takes a random walker to a neighbor v 2 V of the starting node i 2 V , and then it has to reach the node j from there. The matrix of first-hitting times is not symmetric, hij ¤ hj i , even for a regular graph. However, a deeper triangle symmetry property has been proved in [10] for random walks defined by the transition operator (13.11). Namely, for every three nodes in the graph, the consequent sums of the first-hitting times (13.19) in the clockwise and in the counterclockwise directions are equal, hij C hj k C hki D hi k C hkj C hj i : It is clear that the first-passage time to the node i is nothing else but

(13.21)


.i/ D

X

371

˝ ˛ j hj i D hj i ;

(13.22)

j 2V

the average of the first-hitting times hj i over all nodes j from which the random walk starts. In order to find the relations between the first-passage times to nodes, it is convenient to use the spectral representations of the quantities defined above. If we account each node of the graph G accordingly to its degree, i.e., the node i acquires the mass mi D deg.i /, the transition operator of random walks is represented by the symmetric matrix b D 1 1=2 T 1=2 C 1=2 T > 1=2 T 2

(13.23)

with D diag .1 ; : : : N /. The self-adjoint operator (13.23) has real eigenvalues 1 D 1 > 2 : : : N 1, associated with the orthonormal eigenvectors (harmonic maps): k

W V ! S1N 1 ;

k D 1; : : : N;

(13.24)

where S1N 1 is the .N 1/-dimensional unit sphere. For eigenvalues of algebraic multiplicity r > 1, a number of linearly independent orthonormal eigenvectors can be chosen to span the associated linear space. The set of ordered eigenvectors f k gN kD1 forms an orthonormal basis in Hilbert space H.V /. It is important to mention that the first eigenvector 1 belonging to the largest eigenvalue 1 D 1 is nothing else as the positive Perron–Frobenius eigenvector 1;i > 0, which determines the stationary distribution of random walks, 1

b D T

2 1;i

1;

D i

and the Euclidean norm in the orthogonal complement of N X

2 s;i

(13.25) 1,

D 1 i

sD2

is nothing else as the probability of finding the random walker not in i . The spectral representations for the first-hitting time (13.19) and the first-passage time (13.22) equal [28] to hij D j Sij

(13.26)

and j D

N X sD2

1 1 s

2 s;j 2 1;j

;

Sij D

N X kD2

1 .1 k /

k;i

k;j

1;i

1;j

;

(13.27)

372


respectively. The sum Sij in (13.26) is a symmetric nonnegative matrix which elements are nothing else but the expected lengths of subpaths for random walks toward i and j started simultaneously on the graph G [4, 5]. The matrix Sij can be considered as a Gram matrix of the inner product of vectors in the projective space P R.N 1/ associated with the space of transient processes toward the stationary distribution of random walks on G [5]. In [36], it is shown that the matrix Sij gives the spectral representation for the group generalized inverse (a special case of Drazin inverse [2, 16, 30]) of the Laplace operator L D 1 T: The concept of generalized inversion plays an important role in studies of Markov chains, in electrical engineering, linear programming and in many other applications [2, 7]. In particular, it has been shown that virtually all the important characteristics of a finite Markov chain can be determined from the group inverse of the Laplace operator associated with that [7, 30]. Clearly, the generalized inverses can be efficiently used in graph theory [36]. It might be shown (see [36] for details) that .i/ D Si i : By setting the eigenvalue problem for Sij , S'k D k 'k ;

(13.28)

we find that all its eigenvalues, 0 < 1 : : : < N ; are all positive, but the minimal one 0 D 0, which arises due to the fact that the first passage times to vertices of the same graph are linearly dependent. Other eigenvalues k > 0, k D 2; : : : ; N are the characteristic first-passage times at which the first-passage time probability distributions [35] are defined by ki 'k2i , ki D Pr f.i/ D k g

(13.29)

and satisfy the natural normalization condition: X ki D 1:

(13.30)

i 2V

The distribution of the first-passage time (13.29) over nodes would give us important information about the spatiotemporal structures of Markov processes defined on the graph. In contrast to the stationary distribution (13.1), they may not be uniform even if the graph is regular. First-passage time distributions have relevance for the spread of sexually transmitted diseases in a social network, or the spread of computer worms through the Internet [35]. It is remarkable that the largest eigenvalue N is nothing else, but the maximal characteristic first-passage time in the graph, by which all of its most structurally isolated nodes are visited at least once. In concern with that, it is interesting to mention the result of [1] relating the stationary distribution of random walks (13.1) on a discrete set V with its covering time (or the coupon-collector’s time), ( T .!/ D min n W

n [ mD0

) fXm .!/g D V

(13.31)


373

required a random walker visits all nodes in the graph. Given the stationary probability of random walks (13.1) on the set V , the typical value of T is the unique solution of X exp 0i t D 1: (13.32) i 2V

In particular, if jV j D N , t N log N with equality iff 0 is uniform (i.e., the graph G is regular).

2.3 Uncertainty of First-Passage Times In Sect. 2.2, we have seen that each vertex i 2 V can be characterized by the probability i to observe a random walker in that at infinite time, as well as by the .N 1/ probabilities ki , k D 2; : : : N , of that the random walker reaches the vertex i at characteristic time k for the first time. The first-passage time distributions can be used to analyze the structure of the graph [4, 5]. While broad distributions k can be considered as an evidence of the structural homogeneity of the graph with respect to random walks on the time scale k , the localization of that at certain nodes implies that the spreading of random walkers is strongly determined by the graph structure. To measure the uncertainty associated with the first passage times to nodes of the graph at different times k , we may use the first-passage time entropy rates: X X Hk D ki Tij log2 Tij : (13.33) i 2V

j 2V

The difference between (13.12) and (13.33) is that we use the first-passage distribution k instead of the stationary distribution (13.1). The parameter Hk quantifies the uncertainty of random paths which pass through a certain node of the graph at time k . It is important to stress that in general Hk ¤ H , and moreover it seems intuitive that Hk would decay with k and may be H > Hk for some k > 0 , in particular. Indeed, if we demand the first visit of nodes by random walkers occurs at some time, the uncertainty of random paths would be much smaller than that given by (13.8). We are also interested in the difference between the probability distributions k at different characteristic first-passage times k . In probability theory and information theory, the Kullback–Leibler divergence [12, 26], X k k log2 (13.34) K .k ; l / D l i 2V (also known as the information divergence, information gain, or relative entropy) is a noncommutative measure of the difference between two probability distributions

374


k and l . This divergence measures the expected difference in the number of bits required to code samples from one probability distribution when using a code based on another distribution. The relative entropy (13.34) is always nonnegative and is zero if and only if both probability distributions are equal. However, (13.34) does not satisfy the triangle inequality and is not symmetric, so that it is not a distance.

2.4 Random Walks with Random Absorption of Walkers The simplest random walk defined in Sect. 1 and discussed in Sects. 2.1–2.3 is by no means the unique stochastic process that can be defined on an undirected graph. In view of multiple applications, we consider a family of discrete time random walks defined on an undirected graph with random absorption of walkers at the graph nodes, i.e., when the graph G and the random walks defined on G constitute a stochastic dynamical system with nontrivial ergodic properties which can be characterized by some distribution of the walking times P .t/. In the primitive model with absorption, a random walker concludes the travel at any node with some probability ˛ > 0, but keeps on walking with the probability 1 ˛. If the number of walkers during the walk is conserved, and walkers may start wandering again sometimes after being absorbed, such a model is equivalent to lazy random walks, in which the walker changes the site with some probability ˇ > 0, but stays with the probability 1 ˇ [4]. In more complicated models, the absorption of random walkers can be driven by some independent stochastic process, even by a non-Markov one. The model proposed below is based on our previous works on systems driven by non-Markov stochastic processes at a threshold of instability [18] and evolutionary algorithms for generating scale-free graphs [3]. 2.4.1 Model Let us suppose that the instantaneous state of the random walker can be characterized by a real number x 2 Œ0; 1. Another real number y 2 Œ0; 1 plays the role of an absorption threshold pertinent to the node: the random walker is absorbed by the node if x y, but remains active otherwise. In the simplest case, we suppose that the own dynamics of random walkers is purely stochastic and determined by some probability distribution function (pdf): F .u/ D Prfx < ug: In an analogous way, we suppose that the value of threshold y determining the “absorption policy” at the node is also a random variable distributed over the interval Œ0; 1 with respect to some other probability distribution function: Q.u/ D Prfy < ug:


375

In general, F and G are two arbitrary left-continuous increasing functions satisfying the normalization conditions, F .0/ D Q.0/ D 0; F .1/ D Q.1/ D 1: Given a fixed real number 2 Œ0; 1, we define a discrete time random walk on an undirected graph G.V; E/ in the following way. At time t D 0, a random walker is supposed to be located at some initial node X0 D i , and its dynamical variable x is chosen with respect to pdf F , while the absorption variable y at the node is chosen with respect to pdf Q. If x < y, the random walker is not absorbed at the node i and chooses its next location j i uniformly among all neighbors of the node i available in the graph G, and the process goes to time t D 1. At time t 1, the following events happen: 1. With probability , the random variable x is chosen with pdf F – the state of the walker may change; and the absorption threshold y at the node of arrival is chosen with pdf Q – the absorption policy may also change. Otherwise, 2. With probability 1 , the random variable x describing the state of the random walker is chosen with the pdf F , but the absorption threshold y at the node of arrival j i keeps the value it had at time t 1, i.e., the same as at i . If x y, the walk ends at the node of arrival j ; if x < y, the walk continues and goes to time t C 1. Eventually, at some step t, when the state variable x exceeds the absorption threshold value y, the walk stops, and the walker is supposed to be absorbed at the node of residence. The integer value t D t acquired in this random process limits the length of walk. In the next trials, the walk starts again from a randomly chosen site, when x < y: Provided the probability distributions F and Q are given and the control parameter is fixed, the distribution of the durations of walks P .tI F; Q/ has been thoroughly investigated in [18]. Even if in our model the state variable x is treated as a random variable, what is really important is the corresponding pdf F . It is possible to treat x as a deterministic dynamical variable defined by the iterated images of a map of the interval Œ0; 1. In this case, we may assume the existence of an invariant ergodic (Bernoulli) measure dF , for which x is a generic orbit. The stochastic process defined above can be considered as a toy model for trading and exchange, in which a deal is concluded when the price of offer meets the price of demand, while the probability distributions F and Q describe the price elasticity of demand and supply.

2.4.2 Distribution of the Duration of Walks with Random Absorption We introduce the generating function of P .t/ such that: PO .s/ D

1 X tD0

t

s P .t/;

ˇ 1 d t PO .s/ ˇˇ P .t/ D ˇ tŠ ds t ˇ

(13.35) sD0

376


and define the following auxiliary functions: Z K.n/ D

1 0

F .u/n dQ.u/;

ıK.n/ D K.n/ K.n C 1/;

p.l/ D l K.l C 1/; q.l/ D .1 /l K.1/l1 ; r.l/ D l Œ ıK .l C 1/ C .1 / K.l C 1/ ıK.0/ ; D ıK.1/ C .1 / K.1/ ıK.0/:

for l 1; for l 1; for l 1;

p.0/ D 0; q.0/ D 0; r.0/ D 0; (13.36)

Then we find PO .s/ D ıK.0/ C s C

s ŒrO .s/ C p.s/ O q.s/ O C K.1/q.s/ O 1 p.s/ O q.s/ O CK.1/ q.s/O O r .s/ ;

(13.37)

where p.s/; O q.s/; O rO .s/ are the generating functions corresponding to p.l/; q.l/; r.l/; respectively. In the marginal cases D 0 and D 1, the probability P .t/ can be readily calculated, PD0 .t/ D K.1/t ıK.0/;

PD1 .t/ D ıK.t/:

(13.38)

The above equation shows that in the case of D 0; for any choice of the pdf F and Q, the probability P .t/ decays exponentially: Z PD0 .t/ D

1 0

t Z dQ.y/ F .y/

0

1

dQ.y/ .1 F .y// :

(13.39)

In the opposite case D 1; many different types of behavior are possible, depending upon the particular choice of F and Q. To estimate the upper and lower bounds for P .t/ for any , one can use the fact that: K.1/n K.n/ K.1/

and

0 ıK.n/ K.1/;

n 2 N:

Then the upper bound for K.n/ is trivial, since 0 F .u/ 1 for any u 2 Œ0; 1. The upper bound for K.n/ exists if the interval of the random variable u is bounded and therefore can be mapped onto Œ0; 1 (as a consequence of Jensen’s inequality, and of the fact that the function u W! un is convex on the interval 0; 1Œ for any integer n). The calculation given in [18] allows for the following estimation for the upper bound, P .t/ t ıK.t/ C .1 /K.1/ ıK.0/ Œ C .1 / K.1/t1 o n C K.1/ Œ C .1 / K.1/t1 t1

(13.40)


2/(t+1)/(t+2)

h= 1 h = 0.7

In P(t)

Fig. 13.1 The distributions of the duration of walks for the uniformly distributed variables x and y are plotted on a loglog scale for different values of the tuning parameter . P .t / decays exponentially for D 0, but asymptotically follows the power-law / t 2 as D 1 – consistently with the analytical result (13.42)

377

2-t

h = 0.5 h= 0

t and for the lower bound: P .t/ t ıK.t/ C .1 / K.1/t ıK.0/ D t PD1 .t/ C .1 /PD0 .t/ :

(13.41)

We thus see that, for any 0 < 1, the decay of distribution P .t/ is bounded by exponentials. Furthermore, the bounds (13.40) and (13.41) turns into exact equalities, in the marginal cases D 0 and D 1. The simpler and explicit expressions can be given for P .t/ provided the densities are uniform dF .u/ D dQ.u/ D d u for all u 2 Œ0; 1. Then the equations (13.38) give: 2 : (13.42) PD0 . t / D 2 t; PD1 . t / D .t C 1/.t C 2/ For the intermediate values of , the upper and lower bounds are: 2 t 1 C .t C 1/.t C 2/ 2t

P. t /

1C 2

t

:

(13.43)

The distributions of the duration of walks for the uniformly distributed variables x and y for different values of the tuning parameter are presented in Fig. 13.1. 2.4.3 Transition Probabilities of Walks with Random Absorption Given a random walk with random absorption of walkers defined above, we denote the probability of transition from i to j in t > 0 steps by: .t / pij D P .t/ Tt ij ;

(13.44)

378


where T is the transition matrix of random walks (13.11) and P .t/ is the distribution of the durations of walks. It is important to mention that in contrast to (13.15), the transition probability (13.44) does not correspond to a Markov process, and therefore cannot be calculated simply as a power of a matrix. The matrix T has largest eigenvalue 1, with the left eigenvector (the stationary distribution of random walks) and the right eigenvalue 1, with the right eigenvector 1 D .1; 1; : : : ; 1/. Consider the symmetric matrix b T (13.23) which can be written in a spectral form: N X b T D n n n> : (13.45) nD1

Now we have Tt 1=2 Tt D 1=2 b D

N X

tn 1=2

n

> n

1=2

(13.46)

nD1

D C

N X

tn 1=2

n

> n

1=2 ;

nD2

where D diag .01 ; : : : 0N /, and the probability of transition (13.44) is given by: .t /

pij D i P .t/ C

N X nD2

s

kj ki

ni

> nj

tn P .t/:

(13.47)

In concern with (13.47), we are interested in the absorption property of a node – the total probability that a random walker is absorbed at the node j 2 V . 2.4.4 Absorption Probability of Random Walks The similar questions arise in many practical situations and therefore can be of essential importance for applications. Let us suppose for simplicity that the densities of distributions describing the elasticity of prices in the model are uniform, dF .u/ D dQ.u/ D d u for all u 2 Œ0; 1. Such a model can be considered as the simplest example of an unstructured market, in which a customer represented by a random walker explores the complex trading area described by some undirected graph G. While in a trading place, the customer proposes a price x 2 Œ0; 1 for a good randomly and concludes the deal, when the value y 2 Œ0; 1 assigned randomly to the good by a seller meets the proposed price. What is the probability j that the deal is concluded in the trading place j 2 V ? From (13.44), it is obvious that:


j D

XX

.t /

pij D j C

t 1 i 2V

X

s

kj ki

i 2V

N X

X

> nj

ni nD2

tn P .t/;

379

(13.48)

t 1

P where we have taken into account that t 1 P .t/ D 1: It is important to mention that j as defined in (13.48) can be naturally related to the Gram matrix (13.27), ! X f (13.49) S j D j 1 C ij ; i 2V

f in which S ij is the symmetric matrix coinciding with the Gram matrix Sij (13.27) when P .t/ 1: It is clear that while the leading term in j is determined P by the f local property of the node (its connectivity), the structure factor 1 C i 2V S ij describes the possible corrections to j due to the role the vertex j 2 V plays for the entire graph. Below, we show that at Pleast for well-connected graphs, with no f bottlenecks, the structural amendments i 2V S ij may be vanishingly small for any choice of the tuning parameter 2 Œ0; 1: It follows from (13.42) that: j . D 0/ D j C

X i 2V

s

N X

kj ki

ni nD2

> nj

n 2 n

(13.50)

and X

s

N kj X ki

.1 n / Œn C log .1 n / : 2n i 2V nD2 (13.51) For some graphs, the probability (13.48) to conclude the deal at the given node i 2 V may depart essentially from the stationary distribution depending upon the location of the node with respect to the entire structure of the graph, the plasticity of prices – F and Q, as well as the revaluation rate 2 Œ0; 1. It is obvious that such a “commercial attractiveness” quantified by the average fraction of deals concluded at a given place varies from one graph vertex to another, so that we can use the Shannon entropy (sometimes known as self-information) as a measure of its uncertainty, j . D 1/ D j C 2

ni

H .F; QI / D

> nj

X

i log2 i ;

(13.52)

i 2V

as usual, we suppose that 0 log2 0 D 0. It is clear that if all sites in the graph are equivalently attractive (the situation of maximum uncertainty), i D 1=N; then the value of entropy (13.52) reaches the maximum max H D N log2 N: Another limit case is that of a graph for which i ! 1 for some site i 2 V (which do not necessary coincide with a hub, deg.hub/ deg.j /, 8j ¤ hub), while j ' 0 for

380


other sites – then the entropy (13.52) tends to zero. For well-connected graphs, the stationary distribution is responsible for the leading contribution to i , and therefore we can use the P approximative representation (13.49) in (13.52) supposing that the f structure factor j 2V S ij is very small, so that: H .F; QI /

X

i log2 i

i 2V

1 X Xf i Sij : ln.2/ i 2V j 2V

(13.53)

2.5 Biased Random Walks on Undirected Graphs We conclude our brief survey of random walks on undirected graphs with the biased random walks Y D ŒY0 ; Y1 ; : : : Yt ; t 2 N; starting from some random location Y0 D y0 and being characterized by the probability transition between adjacent nodes i j defined by: .B/

Tij

mj Aij ; s2V ms Ai s

D P

(13.54)

where mj is the mass assigned to the vertex i 2 V . It is clear that the usual random walks (13.11) correspond to the choice mi D 1 uniformly for all nodes of the graph. To be concrete, we consider the shortest path random walks introduced in [5], in which mi is nothing else, but the total number of shortest paths between all pairs of vertices in the graph that pass through the vertex i 2 V . In such a walk, among all places adjacent to its current location, a random walker always prefers to move into those which occur on many shortest paths in the graph. In other words, with higher probability, the random walker chooses those vertices that are characterized by the higher betweenness centrality, C.i / D

f#shortest paths through i g f#all shortest pathsg

(13.55)

than those that do not. It is important to stress that the biased transition operator (13.54) does not imply that a node being of low betweenness centrality would be never visited by the walker, but that such a visit is less probable. The crucial difference between biased and unbiased random walks is that while the transition operator (13.11) respects the structure of the graph as captured by its automorphism group [4], the shortest path strategy defined by (13.54) does not. Betweenness (centrality) is one of the centrality measures used to quantify the property of a vertex within the graph. Betweenness is, in some sense, a measure of the influence a node has over the spread of information through the network. Vertices that occur on many shortest paths between others have higher betweenness


381

than those that do not. In theory of complex networks, it is used as a measure that captures how often in average a node may be used in journeys from all places to all others in the transport network. The betweenness centrality (13.55) is essential in the analysis of many real-world networks and social networks, in particular, but costly to compute. The Dijkstra algorithm and the Floyd–Warshall algorithm [11] may be used to calculate C.i /. It is obvious then that the random walk Y defined by (13.54) constitutes a time reversible irreducible Markov chain and may be related to various practical studies concerning the city and intercity routing problems such as the travelling salesman problem, in which the cheapest route is searched [13]. They are also related to pedestrian surveys performed in the framework of space syntax research giving an evidence in favor of that people, in general, prefer to move through the more central (integrated) places in the city [22]. Indeed, the shortest path strategy introduced above is only one among infinitely many other strategies that walkers – whether they are random or not – would follow while surfing through the city. Given a set of positive masses mi > 0 characterizing the attraction of a particular place in the city, one can define the correspondent biased walk by (13.54). However, none of them actually fit the set of graph automorphisms, but one defined by (13.11), with mi D 1: The stationary distribution of the biased random walk defined by (13.54) is the betweenness centrality of a node C.i / defined by (13.55), and therefore the recurrence time to a node in the biased walk given by: .B/

ri

D

1 C.i /

(13.56)

and may depart substantially from the return time in fair random walks (13.2). The key observation related to the stationary distributions (13.1) and (13.55) of random walks defined on graphs is that sometimes a highly connected node (a hub) can have a surprisingly low betweenness centrality and vice versa – the local and global properties of nodes are not always positively correlated. In fact, a graph can be characterized by a certain discrepancy between connectivity and centrality of vertices in that. Such a part–whole relationship between local and global properties of the nodes is known in space syntax theory [22] as intelligibility of urban patterns [4, 5]. The adequate level of intelligibility is proved to be a key determinant of the human behavior in urban environments encouraging peoples way-finding abilities [24]. In order to measure the uncertainty between the connectivity and betweenness centrality of nodes in the graph, we can again use the standard Kullback–Leibler divergence (13.34), X i D . j C/ D i log2 : (13.57) C.i / i 2V

382


3 Entropies of Urban Environments: A Case Study In this section, we define the random walks we discussed in the previous section and study the entropies pertinent to them on the spatial graphs of several urban patterns. “We shape our buildings, and afterward our buildings shape us,” said Sir Winston Churchill.1A belief in the influence of the built environments on humans was common in architectural and urban thinking for centuries [8, 17, 27, 33]. The city determines human behavior by spatial organization of a place that creates new rules for how people move and meet others by chance [23]. Spatial configuration promotes people’s encounters as well as making it possible for them to avoid each other, shaping social patterns [31].

3.1 Urban Patterns Under Scrutiny While in a complex environment, the standard path integration process that governs mechanisms of space perception, path integration, and spatial memorization crucial for efficient navigation in mammals apparently breaks down, so that information provided about distance and direction of motion may be confusing [19, 21, 29, 34]. For thousand years, in na¨ıve geography, people intuitively overcame this difficulty by breaking complex environments into individual spatial elements (zones) and consequent understanding of adjacency relationships between them, before any precise distance measurements became possible. Many heuristic methods have been proposed to identify individual zones in space syntax theory [22]. The toponimical approach in which all interconnected spatial locations that share the same geographical name (of a street, or a square) are assigned to the same spatial identificator [5, 24, 38] seems adequate, since the very fact of naming reflects the individual role the site plays in spatial experience in humans intervening in the city. Supposing the inherent mobility of humans and alikeness of their spatial perception aptitudes, one might argue that common spatial experience of the urban environment can be organized in the form of a universally acceptable complex network of intersecting places of motion and then can be encoded by a spatial graph subjected to the analysis based on graph theory [5]. In particular, we have studied the spatial graphs of the compact urban patterns (see Fig. 13.2) of two medieval German cities (the downtown of Bielefeld in Westphalia and Rothenburg ob der Tauber in Bavaria); the webs of city canals in Venice and in Amsterdam, and the modern urban development of Manhattan, a borough of New York City planned in grid. In view of comparison, we also investigate some statistical properties of random walks defined on the Petersen graph, a regular graph, ki D 3, i D 1; : : : 10, consisting of 10 nodes and 15 edges, P i ki D 30. 1

Famous Quotations/Stories of Winston Churchill at http://www. winstonchurchill. org.


383

Fig. 13.2 The route schemes of (a) the downtown of Bielefeld; (b) Rothenburg ob der Tauber; (c) the Petersen graph; (d) the canal web in Venice; (e) the canal network in Amsterdam; (f) the street grid in Manhattan Table 13.1 Some features of the compact urban patterns we study

Urban pattern

N

M

D

P

Rothenburg ob d.T. Bielefeld (downtown) Amsterdam (canals) Venice (canals) Manhattan

50 50 57 96 355

115 142 200 196 3;543

5 6 7 5 5

10;861 10;938 15;960 34;762 415;260

Some features of the compact urban patterns mentioned in the present section are given in Table 13.1, where N denotes the total number of places of motion (streets, squares, or canals) in a city, and M indicates the number of crossroads and junctions. The distance between two places of motion, A and B, is the length of the shortest path connecting them, i.e., the minimal number of other places one should cross while travelling from A to B, or vice versa. The total number of shortest paths P in the urban pattern is given in the utmost right column. The diameter D of a city graph is the distance between the two places of motion which are furthest from each other.

3.2 Statistics of Random Walks in the Petersen Graph and Urban Environments In Fig. 13.3, we have presented the comparative diagram of entropy rates (13.12) for random walks defined by (13.11) on the spatial graphs of five compact urban patterns and the Petersen graph.

384


Fig. 13.3 The comparative diagram of entropy rates (13.12) for random walks defined by (13.11)

Interestingly, the average dimensions H D hıi i of random walks (13.11) defined on spatial graphs of cities which are either organic or had experienced the organic phase in their developments are close to 2, so that we can say that their urban space, in average, is almost planar [5]. It is worth a mention that the modern street pattern in Manhattan apparently in average seems to be three-dimensional, for random walks. In contrast to all of them, the Petersen graph looks relatively simple with its effective averaged space dimension a little bit above 1. Discovering of important nodes and quantifying differences between them in a graph are not easy, since the graph, in general, does not possess the structure of Euclidean space. Despite a plenty of applications that the first-encounter properties and the first-passage times in particular have in networks [35], surprisingly, they have not been “noticed” by graph theorists yet. In order to draw the attention of readers to the possible applications of the first-passage time distributions in graph theory, we discuss them for different graphs in more details. The Petersen graph constitutes a notorious example in theory of complex network since the graph is regular, and therefore its nodes can be distinguished by none of the standard methods. The stationary distribution of random walks on the graph nodes is .Pet/ uniform, i D 0:1, and the first-passage time to each of nodes is equal to D 9:9 steps. A deeper insight into the structure of the Petersen graph (see Fig. 13.2c) can be obtained while analyzing the first-hitting times between nodes, hij defined by (13.19). It immediately becomes clear that while a random walker started at the node #1 is expected to visit any peripheral node (#2; 3; 4; 5) and then returns back into #1 for the first time in 18 random steps, it takes on average 24 random steps to visit any node in the central component of the graph (#6; 7; 8; 9; 10). Therefore, we


385

Fig. 13.4 The kernel density plots for the distributions k (13.29) of characteristic first-passage times k

can conclude that the Petersen graph contains two components – the periphery and the core – whose nodes appear to be as much as a quarter more isolated in between groups (18 random steps vs. 24 random steps) than those belonging to the same group. The characteristic first-passage times k (13.28) for the Petersen graph are I D 4:286 and II D 30, both with multiplicity 5. The random walker starting at some node of the Petersen graph is expected to reach any other node of the same group in I (less than five steps) steps, otherwise any node of the alternative group P can be reached equiprobably by II D k steps. The five-dimensional linear i i space correspondent to each of the characteristic first-passage times indicate that random walks can only help us in distinguishing between two structurally different groups of vertices – the central one (#6; 7; 8; 9; 10) and the periphery (#1; 2; 3; 4; 5) – however, due to the exceptional symmetry of the Petersen graph, random walks give us no clues to distinguish between nodes within a group. In general, the distributions k (13.29) of characteristic first-passage times k are different for different graphs depending both upon its structure and size (see Fig. 13.4). In order to compare the distributions of k for different urban patterns, we have implemented the method of kernel density estimations (or [32] window method). Kernel density estimation works by considering the location of each data point and replacing that data point with a kernel function which has an area of one (see [39]). The kernels are then individually summed up over all data points and normalized, so that the estimate is a probability density function. In (13.33), we have introduced the first-passage time entropy rates Hk to measure the uncertainty of paths that pass at time k through the certain nodes of the graph with the probabilities ki . For the Petersen graph, all first-passage time entropy rates Hk 1:17 for all k .

386


Fig. 13.5 The first-passage time entropy rates Hk vs. the characteristic first-passage times k for the five compact urban patterns (see Table 13.1). The upper curve shows the decay of Hk with k in Manhattan, the rest curves represent those for the organic cities

In contrast to those of Petersen graph, for the spatial graphs of urban environments, the first-passage time entropy rates depend upon k essentially (see Fig. 13.4). Apparently, the first-passage time entropy rates Hk demonstrate similar behavior with k for all organic cities we studied. In Fig. 13.4, we have seen that the distribution of the characteristic first-passage times in the urban pattern of Manhattan is relatively smooth in comparison with those in the organic cities. This is beyond any doubt a fingerprint of the regular structure of the street array on the island. Interestingly, the decay of Hk with k in Manhattan (depicted by the upper curve in Fig. 13.5) may be relatively well fitted by a power-law Hk / k1 , for some > 0. The spatial network of Venice that stretches across 122 small islands between which the canals serve the function of roads is constituted by 96 canals. In March 1516, the Government of the Serenissima Repubblica issued special laws, and the first Ghetto of Europe was instituted in the Cannaregio district, the northernmost of the city. The Ghetto existed for more than two-and-a-half centuries, until Napoleon conquered Venice and finally opened and eliminated every gate (1797). Despite the political and religious grounds for the ghettoisation of these city quarters had disappeared, these components are still relatively isolated from the major city canal network that can be spotted by estimating the first-passage time probability distributions in the network of Venetian canals (see Fig. 13.6), which can be used as a measure of its accessibility from other canals in the canal network.


387

Fig. 13.6 The first-passage time probability distributions in the network of 96 Venetian canals

Fig. 13.7 The first-passage time probability distributions for the 355 streets in Manhattan

The probability distribution of the first-passage times allows us to explore the connectedness of the entire canal network in the city. In particular, if the graph contains either groups of relatively isolated nodes, or bottlenecks, they can be visually detected on the first-passage times probability distributions (see Fig. 13.6). This method can facilitate the detection of urban ghettos and sprawl [5]. It is remarkable that while most of the Venetian canals can be reached for the first time from everywhere by 300 random steps, the canals surrounding the Venetian Ghetto jump above the average up to from 600 to 800 random steps. The character and development of Manhattan, the acknowledged heart of New York City, are essentially shaped by geography – Manhattan had been only linked to the next boroughs by bridges and tunnels at the end of the nineteenth century. In Fig. 13.7, we have presented the probability distributions of the first-passage times for 355 streets in the street array of Manhattan.

388


Fig. 13.8 The first-passage time probability distributions over the Venetian canals at three characteristic first-passage times 2 10:09, 4 26:78, and 8 51:07 consecutively

While most of the locations in Manhattan may be reached for the first time in 2,000 random steps or less, there are some isolated segments in the city mainly allocated in the Spanish and East Harlems. They are characterized by the longest expected first-passage times from 5,000 to more than 7,000 of random steps on the spatial graph Fig. 13.7. Structural isolation of these locations is fostered by the unfavorable confluence of many factors such as the close proximity to Central Park, the boundedness by the strait of Harlem River separating the Harlem and the Bronx, and the remoteness from the main bridges (the Triborough Bridge, the Willis Avenue Bridge, and the Queensboro Bridge) that connect the boroughs of Manhattan to the urban arrays in Long Island City and Astoria. Being interested in the spatiotemporal patterns of fair random walks in the complicate spatial graphs of urban environments, we investigate the first-passage time probability distributions in more detail. In Fig. 13.8, we have presented the firstpassage time probability distributions over the Venetian canals at three characteristic first-passage times 2 10:09, 4 26:78, and 8 51:07 consecutively. It is clear that at the beginning of random walks (at 2 ), the first-passage probability is localized at one particular canal (the Grand Canal of Venice) which is characterized by the highest betweenness centrality in the urban canal network of Venice. Later (at 4 ), first-passages are dispersed over all canals of the city. Finally, at 8 the firstpassage probability distribution is again localized between just a few vertices of the spatial graph. The first-passage time to a node is determined by its position in the entire graph that can be seen from the distributions in Fig. 13.8.


389

Fig. 13.9 Some of the first-passage probability distributions over 96 canals of Venice at the characteristic first-passage time V D 195

The Gram matrix Sij defined by (13.27) can have eigenvalues k of multiplicity mk > 1. In such a case, there is no definite first-passage time probability distribution k D 'k2 correspondent to k , but a manifold of probability distributions associated with all linear combinations 'bk D

mk X sD1

.s/

˛s 'k

(13.58)

P k characterized by .mk 1/ independent parameters ˛s 2 Œ0; 1 such that m sD1 ˛s D .s/ 1, and 'k is the sth vector spanning the linear subspace belonging to the same eigenvalue k . Any probability distribution from the manifold described by (13.58) should satisfy the natural normalization condition, and therefore it is easy to see that the independent parameters ˛s are nothing else but the elements of a rotation matrix in Rmk . For example, the Gram matrix Sij for the Venetian canals has the eigenvalue V D 195 steps with mV D 10: The first-passage

multiplicity 2 .s/ 'k , s D 1; : : : ; 10; at the first-passage probability distributions .V / D time V D 195 over 96 canals of Venice are shown in Fig. 13.9. All first-passage time probability distributions belonging to the same characteristic first-passage time V are localized at one and the same set of vertices nV in the spatial graph of Venice (see Fig. 13.9). In Sect. 2, we have proposed to measure the difference between the probability distributions k at different characteristic first-passage times k by the Kullback– Leibler divergence (13.34). Interestingly, the effect of multiple eigenvalues of the

390


Fig. 13.10 The Kullback–Leibler divergence between the probability distributions k at different characteristic first-passage times k of fair random walks on the city canal network of Venice Fig. 13.11 The Kullback–Leibler divergence between the probability distributions k at different characteristic first-passage times k of fair random walks on the grid of 355 streets in Manhattan

Gram matrix Sij (and the eigenvalue V D 195, in particular) can be seen in the matrix plot of the Kullback–Leibler divergences (see Fig. 13.10) as a trench of vanishing divergences between the probability distributions .V / belonging to the eigenvalue V D 195 (s ; s D 54; : : : ; 63) and all other first-passage probability distributions. The reason for that is quite clear: all probability distributions of nodes at the first-passage time V D 195 are localized on nV , so that the probability of a random walker arrives for the first-time at any other node i … nV at time V D 195 is almost zero, as well as the probability of that the random walker reaches any node j 2 nV for the first time at some other time 0 ¤ V . Therefore, most of the contributions into the sum in the Kullback–Leibler divergence at time V D 195 is of the form 0 log 0 0. The similar effect of multiple eigenvalues of the Gram matrix Sij is observed for fair random walks on the grid of 355 streets in Manhattan (see Fig. 13.11).


391

Fig. 13.12 The probability to conclude a deal vs. the first passage to a node for Manhattan streets and Venetian canals (in the log–log scale). The densities of distributions describing the elasticity of prices in the model are taken uniform, dF .u/ D dQ.u/ D du for all u 2 Œ0; 1. The plotted probabilities appear to be insensitive to the actual value of the tuning parameter 2 Œ0; 1

Sociologists think that isolation worsens an area’s economic prospects by reducing opportunities for commerce, and engenders a sense of isolation in inhabitants, both of which can fuel poverty and crime [9]. In Sect. 2.4, we have introduced the simple model for trading and exchange defined on an undirected graph G and have calculated the probability to conclude a deal when the price of offer meets the price of demand at a given vertex of the graph (13.48). In particular, in the simplest case, we supposed that the densities of distributions describing the elasticity of prices in the model are uniform, dF .u/ D dQ.u/ D du for all u 2 Œ0; 1, and then the probability to conclude a deal at the given location i 2 V , for the limit values of the tuning parameter modeling the market dynamics D 0 and D 1; was defined by (13.50) and (13.51), respectively. It is interesting to mention that this probability apparently scales with the first passage times to a vertex that can be seen in Fig. 13.12. Indeed, the probability i is dominated by the stationary distribution i , while the structure factor .1 C P e i 2V S ij / 1, for any choice of the tuning parameter 2 Œ0; 1. The Shannon entropy (13.52) for the distributions of probability to conclude a deal i , for the toy model of trading and exchange defined on the spatial graphs of five compact urban patterns and the Peterson graph, is represented by the diagram Fig. 13.13.

392


Fig. 13.13 The Shannon entropy (13.52) for the distributions of probability to conclude a deal is calculated for the toy model of trading and exchange defined on the spatial graphs of five compact urban patterns and the Petersen graph

Fig. 13.14 The Kullback–Leibler divergence (13.57) between the stationary distributions of the fair random walks and the shortest path random walks

Finally, in Fig. 13.14, we estimate the Kullback–Leibler divergence (13.57) between the stationary distributions of the fair random walks defined in Sect. 1 and the shortest path random walks defined in Sect. 2.5. As we have mentioned above, the Kullback–Leibler divergence can help us to investigate the intelligibility of graphs and the spatial graphs of urban patterns, in particular. It is known that in some graphs the local and global properties of nodes are not always positively


393

correlated: sometimes, a highly connected node (a hub) can have a low betweenness centrality and vice versa. The uncertainty between the connectivity and betweenness centrality of nodes in the graphs we studied is shown in Fig. 13.14. It is obvious that while most of the investigated urban patterns are intelligible, the value of the Kullback–Leibler divergence for the city canal network in Venice jumps far above the others.

4 Conclusions Random walks provide us with an effective tool for the detailed structural analysis of connected undirected graphs exposing their symmetries. In our present work, we have analyzed the different models of random walks – fair and biased – and introduced many relevant entropy measures. The main subject of our study was the first-passage time, the expected number of steps in which a random walker arrives at a given node for the first time starting from an arbitrary node of the graph. In contrast to the stationary distribution of random walks which is thoroughly discussed in many surveys, together with the entropy rate associated with that, the first-passage time probability distributions and their entropy rates have not been discussed in graph theory yet. The entropy rates quantifying the uncertainty of first-passage times to the nodes characterize the entire connectedness of the graph. We have also considered two types of biased random walks, such as the random walk with random absorption of walkers and the shortest path random walks, in which a walker chooses its next destination giving a preference to those vertices with the high betweenness centrality index. The relative entropy derived for the biased random walks helps to quantify the uncertainty between connectivity and connectedness – the local and global properties of nodes in the graph G.

References 1. Aldous, D.J.: Lower bounds for covering times for reversible Markov chains and random walks on graphs. J. Theoret. Probab. 2, 91 (1989) 2. Ben-Israel, A., Greville, Th.N.E.: Generalized Inverses: Theory and Applications, 2nd edn. Springer, New York (2003) 3. Blanchard, Ph., Volchenkov, D.: An algorithm generating scale free graphs. Phys. A Stat. Mech. Appl. 315, 677 (2002) 4. Blanchard, Ph., Volchenkov, D.: Intelligibility and first passage times in complex urban networks. Proc. R. Soc. A 464, 2153–2167 (2008). doi:10.1098/rspa.2007.0329 5. Blanchard, Ph., Volchenkov, D.: Mathematical Analysis of Urban Spatial Networks, Series: Understanding Complex Systems, ISBN: 978-3-540-87828-5. Springer, New York (2008) 6. Boccaletti, S., Latora, V., Moreno, Y., Chavez, M., Hwang, D.-U.: Complex networks: structure and dynamics. Phys. Rep. 424, 175 (2006) 7. Campbell, S.L., Meyer, C.D.: Generalized Inverses of Linear transformations. Dover Publications, New York (1979)

394


8. Cao, X., Handy, S.L., Mokhtarian, P.L.: The influences of the built environment and residential self-selection on pedestrian behavior: evidence from Austin, TX. Transportation 33(1), 1–20 (2006) 9. Chown, M.: Equation can spot a failing neighbourhood. New Scientist 2628, 4 (2007) 10. Coppersmith, D., Tetali, P., Winkler, P.: Collisions among random walks on a graph. SIAM J. Discrete Math. 6(3), 363–374 (1993) 11. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press and McGraw-Hill, New York (2001). ISBN 0-262-03293-7. Chapter 21: Data structures for Disjoint Sets, pp. 498–524 12. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991) 13. Dantzig, G.B., Fulkerson, R., Johnson, S.M.: Solution of a large-scale. traveling salesman problem. Oper. Res. 2, 393–410 (1954) 14. Dehmer, M.: Information processing in complex networks: Graph entropy and information functionals. Appl. Math. Comput. 201(1-2), 82–94 (2008) 15. Dehmer, M., Emmert-Streib, F.: Structural information content of networks: Graph entropy based on local vertex functionals. Comput. Biol. Chem. 32(2), 131–138 (2008) 16. Drazin, M.P.: Pseudo-inverses in associative rings and semigroups. Am. Math. Month. 65, 506–514 (1958) 17. Faskunger, J.: The Influence of the Built Environment on Physical Activity. The Special Report of Swedish National Institute of Public Health, ISBN 978-91-7257-494-6, October 2007 18. Floriani, E., Volchenkov, D., Lima, R.: A system close to a threshold of instability. J. Phys. A Math. Gen. 36, 4771-4783 (2003) 19. Fyhn, M., Hafting, T., Treves, A., Moser, M.-B., Moser, E.I.: Hippocampal remapping and grid realignment in entorhinal cortex. Nature 446, 190–194 (2007) 20. Gomez-Gardenes, J., Latora, V.: Entropy rate of diffusion processes on complex networks. To appear in Rapid Communications, Phys, Rev. E (2009), E-print arXiv:0712.0278v1 [condmat.stat-mech] (2007) 21. Hafting, T., Fyhn, M., Molden, S., Moser, M.-B., Moser, E.I.: Microstructure of a spatial map in the entorhinal cortex. Nature 436, 801–806 (2005) 22. Hillier, B.: The Common Language of Space: A Way of Looking at the Social, Economic and Environmental Functioning of Cities on a Common Basis. Bartlett School of Graduate Studies, London (2004) 23. Hillier, B., Hanson, J.: The Social Logic of Space. Cambridge University Press, Cambridge (1984) 24. Jiang, B., Claramunt, C.: Topological analysis of urban street networks. Environ. Plan. B Plan. Des. 31, 151–162 (2004) 25. Kac, M.: On the notion of recurrence in discrete stochastic processes. Bull. Am. Math. Soc. 53, 1002–1010 (1947). [Reprinted in Kac’s Probability, Number Theory, and Statistical Physics: Selected Papers, pp. 231–239] 26. Kullback, S.: Information Theory and Statistics. Wiley, New York (1959) 27. Ledoux, C.-N.: L’Architecture considérée sous le rapport de l’art, des mœurs et de la législation (1804) 28. Lovász, L.: Random Walks On Graphs: A Survey. Bolyai Society Mathematical Studies 2: Combinatorics, Paul Erdös is Eighty, Keszthely (Hungary), p. 1–46 (1993) 29. McNaughton, B.L., Battaglia, F.P., Jensen, O., Moser, M.-B., Moser, E.I.: Path integration and the neural basis of the cognitive map. Nature (Rev. Neurosci.) 7, 663–678 (2006) 30. Meyer, C.D.: The role of the group generalized inverse in the theory of finite Markov chains. SIAM Rev. 17, 443-464 (1975) 31. Ortega-Andeane, P., Jiménez-Rosas, E., Mercado-Doméenech, S., Estrada-Rodrýguez, C.: Space syntax as a determinant of spatial orientation perception. Int. J. Psychol. 40(1), 11–18 (2005) 32. Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)


395

33. Does the Built Environment Influence Physical Activity? Examining The Evidence, Committee on Physical Activity, Health, Transportation, and Land Use, Transportation Research Board, Institute Of Medicine,TRB Special Report 282, Washington, D.C. (2005) 34. Sargolini, F., Fyhn, M., Hafting, T., McNaughton, B.L., Witter, M.P., Moser, M.-B., Moser, E.I.: Conjunctive representation of position, direction, and velocity in entorhinal cortex. Science 312(5774), 758–762 (2006) 35. Shlesinger, M.F.: First encounters. Nature 450(1), 40–41 (2007) 36. Volchenkov, D.: Random Walks and Flights over Connected Graphs and Complex Networks. Communications in Nonlinear Science and Numerical Simulation, http://dx.doi.org/10.1016/j.cnsns.2010.02.016 (2010) 37. Volchenkov, D., Blanchard, Ph.: Nonlinear diffusion through large complex networks with regular subgraphs. J. Stat. Phys. 127(4), 677–697 (2007) 38. Volchenkov, D., Blanchard, Ph.: Random walks along the streets and canals in compact cities: Spectral analysis, dynamical modularity, information, and statistical mechanics. Phys. Rev. E 75, 026104 (2007) 39. Wasserman, L.: All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics (2005)

Towards an Information Theory of Complex Networks: Statistical Methods and Applications

Statistical Mechanics of Complex Networks

Statistical mechanics of complex networks

Graph Theory and Complex Networks: An Introduction

Handbook of Optimization in Complex Networks: Theory and Applications

Information Theory and Statistical Learning

Information Theory and Statistical Learning

Stochastic Networks: Theory and Applications

Statistical factor analysis and related methods: Theory and applications

Statistical factor analysis and related methods: theory and applications

Handbook of Optimization in Complex Networks: Theory and Applications (Springer Optimization and Its Applications)

System Reliability Theory: Models, Statistical Methods, and Applications, Second Edition

System Reliability Theory: Models, Statistical Methods, and Applications, Second Edition

Dynamics of information systems : theory and applications

Information Theory And Statistical Mechanics, II

Complex Networks

Information theory and statistical mechanics I

Biometrics: Theory, methods, and applications

Artificial Neural Networks and Information Theory

Towards Intelligent Modeling: Statistical Approximation Theory

Probability theory for statistical methods

Introducing Social Networks (Introducing Statistical Methods series)

Introducing Social Networks (Introducing Statistical Methods series)

Methods of noncommutative analysis: theory and applications

Methods of Noncommutative Analysis: Theory and Applications

Methods of Noncommutative Analysis: Theory and Applications

Statistical Mechanics of Complex Networks (Lecture Notes in Physics)

Information-spectrum methods in information theory

Information-Spectrum Methods in Information Theory

Statistical Implicative Analysis: Theory and Applications

Towards an Information Theory of Complex Networks: Statistical Methods and Applications

Statistical Mechanics of Complex Networks

Statistical mechanics of complex networks

Graph Theory and Complex Networks: An Introduction

Handbook of Optimization in Complex Networks: Theory and Applications

Information Theory and Statistical Learning

Information Theory and Statistical Learning

Stochastic Networks: Theory and Applications

Statistical factor analysis and related methods: Theory and applications

Statistical factor analysis and related methods: theory and applications

Handbook of Optimization in Complex Networks: Theory and Applications (Springer Optimization and Its Applications)

System Reliability Theory: Models, Statistical Methods, and Applications, Second Edition

System Reliability Theory: Models, Statistical Methods, and Applications, Second Edition

Dynamics of information systems : theory and applications

Information Theory And Statistical Mechanics, II

Complex Networks

Information theory and statistical mechanics I

Biometrics: Theory, methods, and applications

Artificial Neural Networks and Information Theory

Towards Intelligent Modeling: Statistical Approximation Theory

Probability theory for statistical methods

Introducing Social Networks (Introducing Statistical Methods series)

Introducing Social Networks (Introducing Statistical Methods series)

Methods of noncommutative analysis: theory and applications

Methods of Noncommutative Analysis: Theory and Applications

Methods of Noncommutative Analysis: Theory and Applications

Statistical Mechanics of Complex Networks (Lecture Notes in Physics)

Information-spectrum methods in information theory

Information-Spectrum Methods in Information Theory

Statistical Implicative Analysis: Theory and Applications

Recommend Documents