Spacecraft Reliability and Multi-State Failures: A Statistical Approach

P1: TIX/OSW JWST062-FM P2: TIX JWST062-Saleh February 17, 2011 7:24 Printer Name: Yet to Come Spacecraft Reliabilit...

Author: Joseph Homer Saleh | Jean-Fran?ois Castet

104 downloads 963 Views 9MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

P1: TIX/OSW JWST062-FM

P2: TIX JWST062-Saleh

February 17, 2011

7:24

Printer Name: Yet to Come

Spacecraft Reliability and Multi-State Failures A Statistical Approach

Spacecraft Reliability and Multi-State Failures: A Statistical Approach, First Edition. Joseph Homer Saleh and Jean-François Castet. © 2011 John Wiley & Sons, Ltd. Published 2011 by John Wiley & Sons, Ltd. ISBN: 978-0-470-68791-8



February 17, 2011

7:24


Spacecraft Reliability and Multi-State Failures A Statistical Approach Joseph Homer Saleh And

Jean-François Castet Georgia Institute of Technology, USA

A John Wiley and Sons, Ltd., Publication



February 17, 2011

7:24


This edition first published 2011 C 2011 John Wiley & Sons, Ltd Registered office John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com. The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Library of Congress Cataloging-in-Publication Data Saleh, Joseph Homer, 1971– Spacecraft reliability and multi-state failures : a statistical approach / Joseph Homer Saleh, Jean-François Castet. p. cm. Includes bibliographical references and index. ISBN 978-0-470-68791-8 (cloth) 1. Space vehicles—Reliability. 2. System failures (Engineering) I. Castet, Jean-François. II. Title. TL885.S25 2011 629.47—dc22 2010054208 A catalogue record for this book is available from the British Library. Print ISBN: 978-0-470-68791-8 ePDF ISBN: 978-1-119-99406-0 obook ISBN: 978-1-119-99407-7 ePub ISBN: 978-1-119-97007-1 Mobi ISBN: 978-1-119-97008-8 Typeset in 10/12pt Times by Aptara Inc., New Delhi, India



February 17, 2011

7:24


To BB, In loving memory of the days in Harvard Square JHS To my family, friends and So Young, For their support and understanding, from both sides of the Atlantic J-FC



February 17, 2011

7:24


Contents 1

On time, reliability, and spacecraft 1.1 On time and reliability 1.1.1 Reliability: from the word to the engineering discipline 1.1.2 Brief (pre)history of reliability engineering: the enablers and the catalyst 1.2 On spacecraft and reliability: early studies 1.2.1 Overview of early spacecraft reliability and on-orbit failure studies 1.2.2 Beyond the failure rate emphasis in spacecraft reliability studies 1.3 Book organization

1 1 1

2

Nonparametric reliability analysis of spacecraft failure data 2.1 Introduction 2.2 Database and data description 2.3 Nonparametric analysis of spacecraft failure data 2.3.1 Complete versus censored data set 2.3.2 Empirical reliability function from a complete data set 2.3.3 Kaplan–Meier estimator 2.3.4 Kaplan–Meier plot of satellite reliability 2.4 Confidence interval analysis 2.5 Discussion and limitation 2.A Appendix

9 9 10 11 11 13 14 16 17 20 21

3

Parametric analysis and Weibull modeling of spacecraft reliability 3.1 Weibull distribution: an overview 3.2 Probability plots or graphical estimation 3.2.1 Procedure 3.2.2 Weibull plot and Weibull fit of spacecraft reliability 3.2.3 Advantages and limitations of the graphical estimation method 3.3 Maximum likelihood estimation (MLE) 3.3.1 MLE derivation of a Weibull distribution 3.3.2 MLE Weibull fit for spacecraft reliability

23 24 25 25 25

2 4 6 7 7

26 27 28 30



viii

February 17, 2011

7:24


CONTENTS

3.4

3.5

3.6

Comparative analysis of the spacecraft reliability parametric fits 3.4.1 MLE versus graphical estimation Weibull fits 3.4.2 MLE Weibull versus MLE lognormal fits Finite mixture distributions 3.5.1 Methods for estimating parameters of mixture distributions 3.5.2 The 2-Weibull mixture distribution of spacecraft reliability Comparative analysis of the single versus the mixture distribution Weibull fits

4 Data specialization: statistical analysis of spacecraft reliability by orbit and mass categories 4.1 Introduction 4.2 Data description and mass categorization 4.3 Nonparametric analysis of satellite reliability by mass category 4.4 Parametric analysis of satellite reliability by mass category 4.4.1 MLE of single Weibull fit 4.4.2 Mixture distributions 4.4.3 Failure rates 4.5 Orbit characterization 4.6 Nonparametric analysis of spacecraft reliability by mass and orbit category 4.7 Parametric analysis of satellite reliability by mass and orbit category 4.8 Hypotheses for causal explanations 4.8.1 Testing 4.8.2 Procurement and parts selection 4.8.3 Factors intrinsically related to the design of the spacecraft 4.8.4 Factors related to the space environment 4.A Appendix: Tabular data and confidence interval analysis 4.A.1 Tabular data for the nonparametric reliability results in Figure 4.1 and Figure 4.8 4.A.2 Confidence interval analysis

31 31 33 33 34 36 38

43 43 45 46 48 49 51 54 56 57 60 61 63 63 64 65 67 67 68

5 Spacecraft subsystem reliability 5.1 Spacecraft subsystem identification 5.2 Nonparametric reliability analysis of spacecraft subsystems 5.3 Weibull modeling of spacecraft subsystem reliability 5.4 Comparative analysis of subsystem failures

71 71 72 75 78

6 Time to anomaly and failure of spacecraft subsystems: exploratory data analysis 6.1 Introduction 6.2 Anomaly and failure events

83 83 84



February 17, 2011

7:24


CONTENTS

6.3 6.4

7

8

Distribution of anomalies and failure events by subsystem Time to anomaly and failure of spacecraft subsystems

Multi-state failure analysis of spacecraft subsystems 7.1 Introduction 7.2 Setting the stage: multi-state failure analysis and the state transition diagram 7.3 Nonparametric analyses of spacecraft subsystems’ multi-state failures 7.3.1 Censored data and the Kaplan–Meier estimator 7.3.2 Confidence interval analysis 7.3.3 Nonparametric estimations of the conditional probabilities of transitioning between states 7.4 Parametric analyses of spacecraft subsystems’ multi-state failures 7.4.1 MLE Weibull fit of the nonparametric estimates Pî j 7.4.2 Testing the parametric models of the multi-state failure analysis 7.5 Comparative reliability and multi-state failure analysis of spacecraft subsystems 7.5.1 Gyro subsystem 7.5.2 Thruster subsystem 7.5.3 TTC subsystem 7.A Appendix

Toward survivability analysis of spacecraft and space-based networks 8.1 Introduction 8.2 Overview of survivability and resiliency 8.2.1 On survivability 8.2.2 On resiliency 8.2.3 Comparing survivability and resiliency 8.3 Survivability framework 8.4 Introduction to stochastic Petri nets (SPNs) 8.5 SPNs for spacecraft modeling and survivability analysis 8.5.1 Testing the SPN models 8.5.2 Monte Carlo simulation runs for the SPN models 8.5.3 Results 8.5.4 Limitations A.8 Appendix: SPN model of the space-based network (SBN) in Figure 8.6 and its schematic explanation Epilogue

ix

86 90

97 97 99 101 101 103 103 108 108 109 113 113 115 117 118

123 123 124 125 126 127 128 129 131 138 140 141 144 146 149



x

February 17, 2011

7:24


CONTENTS

Appendix A Geosynchronous communication satellites: system reliability and subsystem anomalies and failures A.1 Part I: System reliability analysis A.1.1 Nonparametric analysis of satellite failure data A.1.2 Confidence interval analysis A.1.3 Parametric fits of geosynchronous communication satellite reliability A.2 Part II: Subsystem anomalies and failures A.2.1 Subsystem health scorecard A.2.2 Comparative analysis of subsystem health scorecard Appendix B Electrical power subsystem: comparative analysis of failure events in LEO and GEO B.1 Introduction B.2 Database, sample analyzed, and classes of failure events B.3 Brief literature review B.3.1 Space environment in LEO and GEO B.3.2 Operational constraints on the EPS in LEO and GEO B.4 Reliability and multi-state failure analyses of the EPS B.5 Comparative analysis of the EPS failure behavior in LEO and GEO B.6 Conclusion

151 151 152 153 155 160 161 164

179 179 180 181 181 182 182

185 193

References

195

Index

201

P1: TIX/XYZ JWST062-01

P2: ABC JWST062-Saleh

February 17, 2011

7:26


1

On time, reliability, and spacecraft1 1.1

On time and reliability

Tempus edax rerum (time, devourer of all things). This exclamation by the Roman poet Ovid is meant as a reflection on the human condition and its ephemeral nature. But for an engineer, this phrase can also take a different, less profound but equally thought-provoking meaning: that things fail in time. Engineering artifacts degrade and fail in time; just how they do so, this particular aspect of their relationship with time, is the realm of reliability engineering.

1.1.1

Reliability: from the word to the engineering discipline

Reliability is a popular concept that has been celebrated for years as a commendable attribute of a person or an artifact (Saleh and Marais, 2006). The Oxford English Dictionary defines it as “the quality of being reliable, that may be relied upon; in which reliance or confidence may be put; trustworthy, safe, sure.” Although many words and expressions in the English language seem to have been coined by or attributed to Shakespeare, it seems we owe the word reliability to another English poet who, along with William Wordsworth, founded the English Romantic Movement, namely, Samuel T. Coleridge (1772–1834). The first recorded usage of the word reliability

1 This

chapter was written in part in collaboration with Karen B. Marais, and it is based in part on an article published in Reliability Engineering and System Safety (Saleh and Marais, 2006).




2

February 17, 2011

7:26


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES

dates back to 1816. In praise of his friend the poet Robert Southey, Coleridge wrote (Coleridge, 1983; our emphasis): He inflicts none of those small pains and discomforts which irregular men scatter about them and which in the aggregate so often become formidable obstacles both to happiness and utility; while on the contrary he bestows all the pleasures, and inspires all that ease of mind on those around him or connected with him, with perfect consistency, and (if such a word might be framed) absolute reliability. From this modest, almost apologetic beginning in 1816, reliability grew into an omnipresent attribute – with qualitative and quantitative connotations – that pervades every aspect of the present-day technologically intensive world. Beyond the etymology of the word, the discipline of reliability engineering emerged in the 1950s, its existence officially recognized in a report by the Advisory Group on Reliability of Electronic Equipment (AGREE): The authoritative announcement of the birth of reliability engineering was provided by the AGREE report on June 4, 1957. The report [. . .] provided all the [US] armed services with the assurance that reliability could be specified, allocated, and demonstrated; i.e., that a reliability engineering discipline existed. (Coppola, 1984) AGREE was jointly established in 1952 between the US Department of Defense and the American electronics industry. Its mission was (1) to recommend measures that would result in more reliable equipment, (2) to help implement reliability programs in government and civilian agencies, and (3) to disseminate a better education of reliability (Coppola, 1984). The AGREE report in 1957 announced the birth of reliability engineering, but how did the discipline come about?

1.1.2

Brief (pre)history of reliability engineering: the enablers and the catalyst

The essential ingredients for reliability engineering are probability and statistics. These constitute the analytical foundation upon which rests this and many other engineering disciplines. Tradition has it that we owe the theory of probability to two Frenchmen, Blaise Pascal and Pierre de Fermat. They established the theory in 1654 in a famous exchange of letters spurred by a challenge posed to Pascal by a French nobleman who had an interest in gaming and gambling (Apostol, 1969). The theory was confined to games of chance until Laplace, in 1812, introduced a series of new techniques – mostly an extensive application of analysis to probability – into probability and statistics and expanded their scope of application to numerous practical problems, other than gaming (e.g., demographics, population estimation, life insurance).



February 17, 2011

7:26


ON TIME, RELIABILITY, AND SPACECRAFT

3

In addition to probability and statistics as essential pillars that enabled the rise of reliability engineering, the idea and practice of mass production – the manufacture of goods in large quantities from standardized parts – is another fundamental ingredient in the development of reliability engineering. Interest in the quality of a product goes as far back in time as humans created artifacts (Duncan, 1974): As far back as the Middle Ages, the medieval guilds insisted on a long period of training for apprentices [. . .]. Such rules were in part aimed at the maintenance of quality. Such craftsmanship, however, while apt to deal with quality issues during the design of a single or small number of artifacts, could not have satisfied the need for quality in a high-volume production environment. Mass production therefore heightened the need for a new discipline capable of dealing with quality issues in high-volume production settings. In response to this quality pressure, first came statistical quality control in the late 1920s and early 1930s with the groundbreaking contributions by a young engineer from Bell Laboratories, Walter Shewhart (1891–1967), then reliability engineering in the mid 1950s. Mass production is traditionally associated with Henry Ford and his Model T car. However, although Ford popularized the concept around 1910, high-volume production of items from standardized parts, that is, mass production, had been achieved many years earlier. For example, during the American Civil War, the Springfield Armory was producing over 300 000 rifles per year for the Union Army in 1863, almost as many as the peak production of the Model T Ford. One key idea at the root of mass production is what came to be called the American System of Manufacturing, or the use of standardized, interchangeable parts. The system was developed by Eli Whitney in the United States in the late 1790s (Maier et al., 2003), although credit for the idea of interchangeable parts goes to a French gunsmith, Honoré le Blanc, who first suggested it in the mid eighteenth century (but did not go very far with it because other gunsmiths saw this idea as a threat to their livelihoods and opposed it). At the onset of World War II, with statistics, in particular the theory of sampling, and mass production well established, reliability engineering was ripe to emerge. The catalyst came in the form of an electronic component, the vacuum tube (specifically the Audion or the triode, which was a major improvement on previous tubes), invented by an American, Lee de Forest, in 1906. The tube, which for all practical purposes initiated the electronic revolution, enabled a series of applications such as radio, television, radar, and others. How did the tube contribute to the birth of reliability engineering? Electronics played a critical role in World War II and contributed to the Allies winning the “wizard war”: The vacuum tube, the active element that made the wizard war possible, was also the chief source of equipment failure. Tube replacements were required five times as often as all other equipments. (Coppola, 1984)



4

February 17, 2011

7:26



It is this experience with the vacuum tubes that prompted the US Department of Defense to initiate a series of studies for looking into these failures after the war. These efforts were eventually consolidated and gave birth to a new discipline, reliability engineering. In short, the catalyst that accelerated the coming of this new discipline was the unreliability of the vacuum tube. By the time the existence of reliability engineering was acknowledged in the late 1950s, attitudes began to change since the unwelcoming response that Walter Shewhart’s Statistical Quality Control work received, and the “deep-seated conviction of American production engineers [. . .] that laws of chance have no proper place among scientific production methods” (Duncan, 1974, citing Freeman, 1936), was slowly being replaced by a better predisposition toward statistics and probability in product design. In a faint echo of Ovid’s reflection on TIME, and highlighting the foundational idea of reliability engineering, C. Raymond Knight (1918–), who contributed to the emergence of this discipline, noted in 1991: It may seem strange today, but at that time [1950s] there was considerable resistance to recognizing the stochastic nature of the time to failure, and hence reliability.2 (Raymond Knight, 1991) After its establishment, reliability engineering evolved in several directions, on the one hand toward increased specialization in its statistical techniques, and on the other hand toward a physics of failure approach and what came to be called structural reliability, which was concerned with the structural integrity of buildings, bridges, and other constructions (Denson, 1998). In addition, reliability improvement programs began to emerge, along with the specification of quantitative reliability requirements, marking the beginning of the contractual aspect of reliability. The story of reliability engineering intersects another major technological development, the emergence of the space industry. In a serendipitous accident of history, these two events, the official birth of reliability engineering and the beginning of the space age with the launch of the first active space system, occurred in the same year, 1957. This book is at the intersection of these two developments and it brings the former, reliability engineering, to bear on the latter, space systems.

1.2

On spacecraft and reliability: early studies

On October 4, 1957, a small beeping satellite, Sputnik, heralded the beginning of the space age. From this humble start, the space industry grew into an impressive 2 At

present, reliability is more formally defined as the probability that an item will perform a required function under stated conditions for a given period of time.



February 17, 2011

7:26



5

$100+ billion industry five decades later. Roughly speaking, around 6500 spacecraft were launched in the five decades after Sputnik. And although the launch rate has been highly variable (Hiriart and Saleh, 2010), a rough estimate would set it at present to around 80 to 100 spacecraft per year. Spacecraft today fulfill a myriad of functions, from defense and intelligence missions (early warning, reconnaissance, etc.), to science missions (Earth observation, interplanetary probes), and communication functions (direct-to-home, fixed satellite services, and mobile satellite services). Spacecraft can cost several hundred millions of dollars to design and launch,3 and as such reliability is essential for these systems. More generally, reliability is a critical design attribute for high-value systems operating in remote or inhospitable environments such as spacecraft or subsea installations. Since physical access to these assets is difficult or impossible, maintenance cannot be relied upon to compensate for substandard reliability (Rausand and Høyland, 2004). As a result, designing high reliability into these systems is an essential engineering and financial imperative. For space systems, statistical analysis of flight data, in particular of actual onorbit (field) anomaly and failure data, would provide particularly useful feedback to spacecraft designers. For example, such analyses can help guide spacecraft testing programs and provide an empirical basis for subsystem redundancy and reliability growth plans. Analyzing spacecraft failure behavior on orbit, and identifying their subsystems’ actual reliability profiles, not their reliability requirements (how they actually degrade and fail on orbit, not how they should or are expected to), can help spacecraft manufacturers prioritize and hone in on problematic subsystems that would benefit most from reliability improvements. Reliability improvements can be achieved through redundancy, increased testing prior to launch, or better design and parts selection, and these efforts would result in a decreased likelihood of spacecraft experiencing failure events. In addition, identifying whether specific spacecraft subsystems experience “infant mortality,” for example, would provide a clear opportunity for spacecraft manufacturers and equipment providers to develop burn-in procedures for weeding out early failures in such subsystems. Statistical analysis of on-orbit failure and spacecraft reliability can also provide important and actionable information to stakeholders other than spacecraft manufacturers. For example, satellite operators may be particularly interested in the reliability profiles of their on-orbit assets, for planning and risk mitigation purposes, and insurers evidently rely on such analysis and information to set up their policy and insurance premiums. The importance of statistical analysis of on-orbit failure data was recognized early in the advent of the space age. The following subsections provide a brief overview of past spacecraft reliability studies. 3 Except for microsatellites, which are typically in the $10–50 million range, and ongoing efforts are seeking to significantly reduce this price tag. Whether useful functions can be performed on orbit below this range remains to be seen.



6

1.2.1

February 17, 2011

7:26



Overview of early spacecraft reliability and on-orbit failure studies

A few years after the launch of the first satellites, statistical analyses of spacecraft reliability and on-orbit failures began to appear. One of the earliest reliability studies, according to Leventhal et al. (1969), was published in 1962, and it analyzed the failure behavior of 16 satellites launched before November 1961 (ARINC, 1962). Over the years, similar analyses were conducted with larger sample sizes or spacecraft populations. For example, Bean and Bloomquist (1968) analyzed the failure behavior of 225 satellites; Timmins and Heuser (1971) and Timmins (1974; 1975) analyzed the failure behavior of 57 satellites; and Hecht and Hecht (1985) and Hecht and Fiorentino (1987) analyzed the failure behavior of some 300 satellites. The present work analyzes the anomaly and failure behavior of 1584 Earth-orbiting satellites launched between January 1990 and October 2008. The choice and impact of the sample size on the statistical results, and their relevance, are discussed in Chapter 2. Early spacecraft reliability studies assumed an exponential distribution and constant failure rate. This assumption, however, was challenged by Timmins and Heuser (1971) who showed that, for their small sample of 57 spacecraft launched from NASA Goddard Space Flight Center, the failure rate was not constant but higher in the early days on orbit: The number of failures per spacecraft were abnormally high for the first 30 days in space. The number of first-day failures departed even more from the longer trend. This finding of spacecraft “infant mortality” and a decreasing failure rate was repeated in subsequent studies (Timmins, 1974; 1975), and led Baker and Baker (1980) to comment that “those spacecraft that last, last on and on,” which in effect reflects for these authors the absence of wear-out failures in spacecraft. Hecht and Hecht (1985) analyzed a different population of spacecraft than the one used in the previous four studies (the 57 NASA spacecraft). Their sample consisted of some 300 spacecraft launched between 1960 and 1984, and covered 96 different space programs. Their analysis also found a decreasing failure rate in their spacecraft sample, and they took issue with the constant failure rate models proposed in the military reliability handbook, MIL-HDBK-217, as unrealistic for system reliability predictions. MIL-HDBK-217 was first developed in 1961 and revised several times afterward. Similar conclusions were advanced by Krasich (1995) and Sperber (1990; 1994) who noted a qualitative agreement in prior studies “that as the mission goes on, risk per unit time to surviving spacecraft decreases.” Some studies explored causal hypotheses for this actuarial result, the decreasing failure rate of spacecraft. Norris and Timmins (1976) for example stated that “a plausible explanation for this decreasing trend is that the data sample includes a wide variety of components, and as the high risk component fail, the remaining units are the ones with lower failure rates.” Baker and Baker (1980) excluded the space



February 17, 2011

7:26



7

environment as a possible cause of this trend by noting that “space itself is not a harsh environment for spacecraft; for if it were, the hazard rate would increase as a function of time as cumulative exposure precipitates failures.” Hecht and Fiorentino (1987) argued for the existence of decreasing failure rate as follows: in terms of spacecraft reliability, that the equipment has survived under the environmental stresses experienced during a period of m years on orbit does not preclude the occurrence of a phenomenon during year m+1 that produces a greater stress and hence lends failure. However, the likelihood that greater stresses will be encountered decreases over successive intervals, and that leads to the decreasing failure rate. The quality of these arguments pertaining to the cause of spacecraft infant mortality is questionable.

1.2.2

Beyond the failure rate emphasis in spacecraft reliability studies

Sperber (1994) suggested that “the causes of the [on-orbit failures and] anomalies are not random overstress or wear-out, but are perhaps weakness in design or execution uncovered in the mission.” His comment echoes an earlier finding by Bean and Bloomquist (1968) that, for the sample they had, that is, 225 spacecraft launched prior to 1968, the “most common cause of spacecraft anomalies is inadequate design, representing nearly 60% of all incidents with assignable causes.” More recent studies revolved around specific spacecraft subsystems. For example, Cho (2005) and Landis et al. (2006) focused on failures in spacecraft power subsystems, Brandhorst and Rodiek (2008) on solar array failures, and Roberston and Stoneking (2003) on failures in attitude control subsystems. Sperber (2002) and Tafazoli (2009) analyzed not just a single subsystem’s failures but the comparative contribution of various subsystems to spacecraft on-orbit failures. And instead of spacecraft subsystems, Bedingfield et al. (1996) focused on spacecraft failures due only to the natural space environment.

1.3

Book organization

In Chapter 2, a statistical analysis of spacecraft failure data is conducted and nonparametric spacecraft reliability results are derived. In Chapter 3, parametric analysis of spacecraft reliability is conducted, and single Weibull as well as mixture distribution models are derived using the maximum likelihood estimation (MLE) method. In Chapter 4, the previously analyzed failure data is specialized first by spacecraft mass category and then orbit type. Nonparametric analysis and parametric reliability models are then derived for these different types of spacecraft. In Chapter 5, the statistical failure analysis is extended to spacecraft subsystems, that is, the analysis is narrowed down from system-level to subsystem-level



8

February 17, 2011

7:26



failures, and reliability results, both nonparametric and parametric, are derived for spacecraft subsystems. Chapter 6 is a turning point in the book. The previous chapters deal with the reliability of spacecraft and spacecraft subsystems. Only two states are considered up to this chapter, operational and failed, and the (sub)systems are analyzed and modeled as being in one of these two states. In reality, engineering artifacts can experience failure events of varying severity, and thus transition from fully operational to various states of partial degradation and failure. Chapter 6 extends the previous analyses of reliability, in its traditional binary-state understanding, to account for spacecraft anomalies and failures of various severities. Partial failures constitute a significant portion of the anomalous events that spacecraft experience on orbit, and as such their analysis provides additional and important information for understanding the spacecraft and subsystems’ failure behavior on orbit. Chapter 6 can be characterized as an exploratory data analysis of failure distributions on orbit, as well as the time to anomaly and failure of spacecraft subsystems. The chapter serves as an easy transition between the formal binary-state understanding of reliability conducted in the previous chapters and the formal multi-state failure analysis in the following chapters. Chapter 7 provides a formal multi-state failure analysis of spacecraft subsystems. And Chapter 8 extends the previous analyses to include considerations of survivability of spacecraft and space-based networks. In addition, Chapter 8 introduces an important tool for the modeling and analysis of stochastic processes, namely, stochastic Petri nets (SPNs), and develops SPN models for the analysis of spacecraft survivability, building on the detailed models of subsystems’ multi-state failures developed in Chapter 7. The analysis and results in each chapter build to some extent on those in previous chapters. As a result, cross-references between chapters are frequent. An effort has been made to facilitate readability by repeating some of the essential material for each chapter. The reader who wishes to go through the whole book in one sitting can easily skip through the overlapping parts. Two appendices are included in this book, and they address specialized topics under the broad theme of this work. Appendix A focuses solely on communication satellites in geosynchronous orbit (GEO). These satellites represent an important segment of the space industry, and, as such, a dedicated appendix for their reliability analysis is provided here. In addition, a health scorecard is developed, summarizing for each subsystem its track record of on-orbit anomalies and failures. Appendix B focuses solely on the electrical power subsystem (EPS) on board spacecraft, and it analyzes the differences in failure behavior of the EPS in low Earth orbit (LEO) and geosynchronous orbit (GEO).



February 17, 2011

10:3


2

Nonparametric reliability analysis of spacecraft failure data1 2.1

Introduction

Reliability is a critical design attribute for systems operating in remote or inhospitable environments such as satellites or subsea installations. Since physical access to these high-value assets is difficult or impossible, maintenance cannot be relied upon to compensate for substandard reliability (Rausand and Høyland, 2004). As a result, designing high reliability into these systems is an essential engineering and financial imperative. For space systems, statistical analysis of flight data, in particular of actual on-orbit (field) failure and anomaly data, would provide particularly useful feedback for spacecraft designers. For example, such analysis can help guide satellite test and screening programs and provide an empirical basis for subsystem redundancy and reliability growth plans. In this chapter, statistical analysis of satellite failure data is conducted from an extensive data set, and nonparametric satellite reliability results are derived. How reliable are satellites, or how reliable have they been? How can on-orbit failure data be analyzed to derive satellite reliability results and bounds or confidence intervals on these results? Do satellites exhibit infant mortality, or other failure behaviors? These are some of the questions addressed in this chapter.

1 This chapter is based on an article written by the authors and published in the Journal of Spacecraft and Rockets (Castet and Saleh, 2009a).




10

February 17, 2011

10:3



Before discussing these points, it is worth addressing a common argument that competitive sensitivity is one reason for the lack of published data and statistical analysis of on-orbit reliability. While this might be true for satellite manufacturers, it is not the case for satellite operators, private or government agencies, whose interests are better served by transparent reliability analysis of spacecraft. Furthermore, satellite manufacturers could also benefit, in the long term, by having spacecraft reliability analyzed and published. Such studies can serve for example as a transparent benchmark to which satellite manufacturers can compare and hence improve their products’ reliability. Such studies, an example of which is provided in this and subsequent chapters, can also constitute a basis and motivation for satellite manufacturers to (re)design their testing and reliability improvement programs. The remainder of this chapter is organized as follows. The database used in this work and the data description are presented in Section 2.2. The nonparametric reliability analysis of satellite failure data is conducted in Section 2.3, and it is followed by the confidence interval analysis in Section 2.4. The chapter concludes in Section 2.5 with a discussion of the results and some limitations. The appendix to this chapter contains tabular data for the Kaplan–Meier reliability results and confidence intervals.

2.2

Database and data description

The SpaceTrak database (see References) was adopted for the purpose of this book. This database is used by many of the world’s launch providers, satellite insurers, operators, and satellite manufacturers. It provides a history of on-orbit satellite failures and anomalies, as well as launch histories since 1957. It should be pointed out that this database is not necessarily “complete” in a statistical sense since some military or intelligence satellites may not have had their failures reported. Similarly, the database cannot be considered “complete” with respect to anomalies or partial failures since satellite operators may not have reported all partial failures, especially the ones that can be recovered from in a timely manner. That said, the database is considered the most authoritative in the space industry with failure and anomaly data for over 6400 spacecraft. The statistical analysis in this work is enabled by, and confined to, the failure and anomaly information provided in this database. The sample analyzed in this chapter consists of 1584 spacecraft. The sample was restricted to Earth-orbiting satellites successfully launched between January 1990 and October 2008. A failure leading to satellite retirement is identified in the database as a Class I failure, that is, a complete failure leading to the loss of the spacecraft; Class I failures are used in this chapter. In addition, as will be detailed later in Chapter 5, 11 spacecraft subsystems are identified in the database. If the cause of a Class I failure is identified and traced back to a particular subsystem, that “culprit” subsystem is noted in the database. When the culprit subsystem whose failure led to the spacecraft failure could not be identified, the failure of the spacecraft is ascribed to an “Unknown” category in the database. This categorization was used for analyzing the relative contribution of each subsystem to the overall spacecraft failures (details are given in Chapter 5).



February 17, 2011

10:3


NONPARAMETRIC RELIABILITY ANALYSIS OF FAILURE DATA

11

Table 2.1 Data collection template and sample data for the statistical analysis of spacecraft reliability (spacecraft are not arranged/shown in chronological order).

Launch date

Failure date (if failure occurred)

Culprit subsystem

Censored time (if no failure occurred)

Spacecraft 1 Spacecraft 2 .. .

Nov. 6, 1998 Mar. 1, 2002 .. .

Nov. 15, 1998 — .. .

TTC∗ — .. .

— Oct. 2, 2008 .. .

Spacecraft 1584

Apr. 6, 2004

Mar. 28, 2006

Mechanisms

—

Sample unit number

∗

Telemetry, Tracking, and Command.

For each spacecraft in the sample, the following information was collected: (1) its launch date; (2) its failure date, if failure occurred; (3) the subsystem identified as having caused the spacecraft failure, hereafter referred to as the culprit subsystem; and (4) the censored time, if no failure occurred. This last point is further explained in the following section, where data censoring and the Kaplan–Meier estimator are discussed. The data collection template and sample data for the analysis are shown in Table 2.1.

2.3 2.3.1

Nonparametric analysis of spacecraft failure data Complete versus censored data set

Censoring occurs when life data for the statistical analysis of a set of items is “incomplete.” This situation occurs frequently in multiple settings (e.g., medical and engineering contexts) and can happen because some items in the sample under study are removed prior to failure or because the test or observation window ends prior to all items failing. By contrast, a life data set is said to be “complete” if one observes the actual time to failure of all the items in the sample under study, that is, if no censoring occurs within the data. Censoring introduces particular difficulties in statistical analysis which, if not addressed and accounted for, can significantly bias the results. There are multiple classifications and types of censoring and different statistical techniques for dealing with them. The two broad classes of censoring are “right censoring,” for which the activation time of a unit, or when it is put in service, is known but its failure time is not observed, and “left censoring,” for which the activation time of a unit is not known. Furthermore, different types of right censoring exist. For example:

r Type 1 censoring qualifies a situation where times to failure are recorded until reaching a predetermined time tend (duration of the observation window).

r Type 2 censoring qualifies a situation where the times to failure are recorded until a predetermined number k of failure occurs.



12

February 17, 2011

10:3



r Type 3 censoring is a combination of Types 1 and 2, stopping the recording of failure times either when t > tend or when the number of failure reaches k, whichever happens first.

r Type 4 censoring, also referred to as random censoring, qualifies a situation where both failure dates and censoring times are stochastic. The reader interested in more extensive detail is referred to three excellent books on the subject: Lawless (2003), Ansell and Phillips (1994), and Meeker and Escobar (1998). The following discussion focuses on the special type of censoring encountered in this study and setting. The sample analyzed here is right censored (random censoring) with staggered entry. This means that: (1) the units in the sample are activated at different points in time (i.e., the satellites are launched at different calendar dates) but all activation times in the sample are known; (2) failure dates and censoring are stochastic; and (3) censoring occurs either because a unit (satellite) is retired from the sample before a failure occurs or because the satellite is still operational at the end of the observation window (October 2008). This situation is illustrated in Figure 2.1. Staggered entries are easily handled by shifting all the activation times to t = 0, which changes the approach, and the x-axis in Figure 2.1, from a calendar-time to a clock-time analysis of satellite reliability. Therefore satellite reliability is investigated as a function of time following successful orbit insertion. Censoring of data requires particular attention. Deriving a reliability function from censored life data is not trivial, and it is important that it is done properly if the

Figure 2.1 Censored data with staggered entry.



February 17, 2011

10:3



13

results are to be meaningful and unbiased. In this work, the powerful Kaplan–Meier estimator (Kaplan and Meier, 1958) is adopted, as it is best suited for handling the type of censoring in the sample. The following subsection reviews how to derive an empirical reliability function from a complete data set, and it is followed by a discussion and derivation of the Kaplan–Meier reliability estimator for right-censored data (Type 4).

2.3.2

Empirical reliability function from a complete data set

Consider n units activated at the beginning of an experiment, and let the experiment run until all units have failed. Let t(1) < t(2) < t(3) < · · · < t(n−1) < t(n)

(2.1)

be the times to failure of all the units arranged in ascending order. Assume that the lifetimes of the units are independent and identically distributed, and that there are no ties in the data set, that is, no two units have exactly the same time of failure (the case with ties is discussed later). The empirical reliability function from this complete data set, Rn (t), corresponds to the number of units still operational at time t given the initial n operating units: Rn (t) =

number of units still operational at time t . n

(2.2)

The following results are obtained for Rn (t): 1. For t < t(1) , no failure has yet occurred, therefore Rn (t) = 1. 2. For t(1) ≤ t < t(2) , only one unit has failed, and n − 1 units remain operational within this time frame. Therefore, Rn (t) =

1 n−1 =1− . n n

3. For t(2) ≤ t < t(3) , two units have failed, and n − 2 units remain operational within this time frame. Therefore, Rn (t) =

2 n−2 =1− . n n

It is thus easy to see the pattern and generalize the result: 4. For t(i) ≤ t < t(i+1) , i units have failed, and n − i units remain operational within this time frame. Therefore, Rn (t) = 1 −

i . n

5. Finally, for t > t(n) , all units will have failed and, in this time frame, Rn (t) = 0.



14

February 17, 2011

10:3


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES 1.0 0.9 0.8 0.7

ΔRn =

Rn(t)

0.6

1 n

0.5 0.4 0.3 0.2 0.1 0.0

t (1) t (2)

t (3)

t (4)

time

t (5)

t (6) …

Figure 2.2 Illustrative empirical reliability function from a complete data set of n units (with no ties).

To summarize, the empirical reliability function from a complete data set of n units is given by ⎧ 1 ⎪ ⎪ ⎪ ⎨ i Rn (t) = 1 − ⎪ n ⎪ ⎪ ⎩ 0

for t < t(1) for t(i) ≤ t < t(i+1)

for all i such that 1 ≤ i < n .

(2.3)

for t > t(n)

Rn (t) is therefore a staircase function with discontinuities at the observed time of failure of each unit, and with a downward jump at each discontinuity of 1/n. An example of such a function is shown in Figure 2.2. When ties are present in the data set, say mj units failing at the same time t(j) , a simple adjustment to (2.3) is required (Rausand and Høyland, 2004). Equation (2.2) remains the basis for calculating the resulting empirical reliability, and the discontinuity in Rn (t) at t(j) is now mj /n, instead of 1/n as in the case with no ties.

2.3.3

Kaplan–Meier estimator

Different presentations of the Kaplan–Meier estimator exist in the statistical literature. In the following, a simple presentation of this tool is provided with an easy notation and indexing consistent with the earlier discussion in this section.



February 17, 2011

10:3



15

Consider n operational units for a start, and, because of censoring, only m times to failure (m < n) are collected. For the time being, assume no ties between failure times and censoring. As done previously, the failure times are organized in ascending order: t(1) < t(2) < · · · < t(m) .

(2.4)

The difference between the current situation and that of a complete data set is that some units may have been removed from the experiment (censored) between two consecutive failure times, and, as a result, the index of time t in the present case, t(i) , is no longer indicative of the number of failures prior to t(i) , as was the case in (2.1) and (2.3). To capture this information, the following parameter is introduced: n i = number of operational units right before t(i) = n − number of censored units right before t(i) − number of failed units right before t(i) .

(2.5)

Then define pˆ i =

ni − 1 . ni

(2.6)

One major contribution of the Kaplan and Meier publication was to note that (2.6) is an estimate of the conditional probability of surviving just past t(i) , or, more precisely, pˆ i estimate of P TF > t(i) + δt|TF > t(i)

(2.7)

where δt is an arbitrary small time interval in which no censoring or failure occurs. The other major contribution by Kaplan and Meier was to note that P(TF > t(i) ) = P(TF > δt) · P(TF > t(1) + δt|TF > t(1) ) (2.8) · P(TF > t(2) + δt|TF > t(2) ) · · · · P(TF > t(i) + δt|TF > t(i) ) and to replace each factor on the right-hand side of (2.8) by its estimate as provided in (2.6). Note that, by the definition of δt, P(TF > δt) = 1. Finally, recall that the reliability function is defined as R(t) ≡ P(TF > t).

(2.9)

Thus the Kaplan–Meier estimator of the reliability function with censored data is derived by substituting (2.6) and (2.9) into (2.8): ˆ = R(t)

all i such that t(i) ≤t

pˆ i =


ni − 1 . ni

(2.10)



16

February 17, 2011

10:3



As a result, the estimated reliability function from censored data, as given by (2.10), is a staircase function with discontinuities at the observed failure times. The estimated reliability function does not change at the censored times. However, unlike the previous case with complete data ((2.3) and Figure 2.2), the jumps at the discontinuities are not equal (1/n) but vary with the number of censored units between two consecutive failures (this is the result of (2.5) and its propagation through (2.6) and (2.10)). The graphic representation of (2.10) is referred to as a Kaplan–Meier plot. When the data set is complete, the Kaplan–Meier estimator is equivalent to the empirical reliability function, that is, (2.3) and (2.10) provide identical results. Ties in the data set are handled in the following manner. Two situations arise: 1. If one faces ties in the failure times, say mi units failing at exactly t(i) , this is referred to as a tie of multiplicity m. Equation (2.6) is then replaced by pˆ i =

ni − m i . ni

(2.11)

2. If a censoring time is exactly equal to a failure time, t(i) , a convention is adopted that assumes censoring occurs immediately after the failure, since a unit that is censored at a given time can survive an infinitely small period past t(i) (Ansell and Phillips, 1994; Meeker and Escobar, 1998).

2.3.4

Kaplan–Meier plot of satellite reliability

The on-orbit satellite reliability from the censored data set can now be analyzed. For the 1584 satellites in the sample, there are 98 failure times and 1486 censored times. The (ordered) failure times are provided in Table 2.2. The data is then treated with the Kaplan–Meier estimator (2.10), and the Kaplan–Meier plot of satellite reliability shown in Figure 2.3 is obtained.

Table 2.2 Failure times (in days) of satellites launched between January 1990 and October 2008. 1 7 64 190 364 1122 1797 2434 3159

1 9 68 197 465 1146 1836 2472 3268

1 12 73 221 515 1167 1967 2577 3455

1 15 79 229 696 1184 2009 2580 3684

2 15 89 237 701 1233 2091 2624 3759

3 16 102 252 713 1256 2097 2702 4192

3 16 107 271 722 1347 2098 2917 4324

4 23 123 309 724 1458 2181 2947 4909

4 36 128 314 787 1551 2191 2963 5043

5 51 131 317 1053 1637 2237 3038 5207

5 53 167 334 1073 1778 2429 3077


February 17, 2011

10:3



17

1.00 0.99 0.98 0.97 Reliability


0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure 2.3 Kaplan–Meier plot of satellite reliability.

Figure 2.3 reads as follows. For example, after a successful launch, satellite reliability drops to approximately 96% after 2 years on orbit. More precisely, ˆ = 0.964 R(t)

for

1.982 years ≤ t < 2.155 years.

Satellite reliability drops to approximately 94% after 6 years on orbit. Beyond 12 years, satellite reliability lies roughly between 90% and 91%. The precise data for this figure is provided in Table 2.A.1, in the appendix to this chapter. Recall that these are actual (field) satellite reliability results, not reliability specifications, which provide a first answer to “how reliable have satellites been?” (between 1990 and 2008). Several trends can be seen in Figure 2.3, the most noticeable one being the steep drop in reliability during the first year of satellite operation, which is indicative of infant mortality. These trends are better captured and further discussed in a parametric analysis of reliability, the focus of the next chapter.

2.4

Confidence interval analysis

In this section, the confidence interval and dispersion of satellite reliability around the Kaplan–Meier estimate are analyzed. The Kaplan–Meier estimator (2.10) provides a maximum likelihood estimate of reliability but does not give information about ˆ i ). This dispersion is captured by the variance or standard the dispersion around R(t deviation of the estimator, which is then used to derive upper and lower bounds for, say, a 95% confidence interval (i.e., a 95% likelihood that the actual reliability will



18

February 17, 2011

10:3



fall between these two bounds, with the Kaplan–Meier analysis providing the most likely estimate). The variance of the Kaplan–Meier estimator is provided by Greenwood’s formula: ˆ i) 2 vâr [R(ti )] ≡ σ 2 (ti ) = R(t j≤i

mj . n j (n j − m j )

(2.12)

The Kaplan–Meier estimator, a maximum likelihood or product-limit estimator, has an asymptotic normal property (Rausand and Høyland, 2004), and, as a consequence, the use of a normal approximation is justified for the construction of confidence ˆ i ). The 95% confidence interval is determined by intervals around R(t ˆ i ) ± 1.96σ (ti ). R95% (ti ) = R(t

(2.13)

Reliability

Although not encountered in the present analysis, some issues may arise while using (2.12) and (2.13), one solution to which is to truncate and confine the values R95% (ti ) to the interval [0, 1]. Another possibility is suggested by Kalbfleisch and Prentice (1980), where the confidence intervals are built on a transformation of the estimator. The reader is referred to Kalbfleisch and Prentice (1980) and Lawless (2003) for more details about alternatives to (2.12) and (2.13). When (2.12) and (2.13) are applied to the data from the sample of 1584 satellites ˆ i ), shown in Figure 2.3 along with the Kaplan–Meier estimated satellite reliability R(t and provided in Table 2.A.1, the 95% confidence interval curves are obtained, as shown in Figure 2.4.

1.00 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92

Nonparametric estimation 95% confidence interval bounds

0.91 0.90 0.89 0.88 0.87 0


Figure 2.4 Satellite reliability with 95% confidence intervals.


February 17, 2011

10:3



19

The tabular data for Figure 2.4 is provided in the appendix of this chapter, in Table 2.A.2. Vertical cuts across Figure 2.4 read as follows, for example: 1. Satellite reliability at t = 1 year will be between 96.1% and 97.8% with a 95% likelihood – these values constitute the lower and upper bounds of the 95% confidence interval at t = 1 year. In addition, the most likely estimate of satellite reliability at this point in time is Rˆ = 97.0%. 2. Satellite reliability at t = 7 years on orbit will be between 92.3% and 95.1% with a 95% likelihood. In addition, the most likely estimate of satellite reliability at this point in time is Rˆ = 93.7%. ˆ i ) increases with time. This increase Notice that the dispersion of R(ti ) around R(t in dispersion can be seen in Figure 2.4 by the growing gap between the Kaplan–Meier estimated reliability and the confidence interval curves. The dispersion is given by (2.14), and its growth over time is clearly seen in Figure 2.5: D(ti ) = upper bound R95% (ti ) − [lower bound R95% (ti )] = 3.92σ (ti ) ˆ i) = 3.92 R(t j≤i

(2.14) mj . n j (n j −m j )

Figure 2.5 illustrates the increasing uncertainty or loss of accuracy of the nonparametric analysis of satellite reliability with time. For example, after 2 years on orbit, satellite reliability is dispersed over a 2% point interval (with 95% confidence), 5.0 4.5 Dispersion (percentage point)


4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0


Figure 2.5 Dispersion of the 95% confidence interval of satellite reliability.



20

February 17, 2011

10:3



whereas after 12 years on orbit, satellite reliability is dispersed over a 3.7% point interval. This is a direct result of the decreasing sample size with time and how it is handled in (2.12) as more satellites fail or are retired.

2.5

Discussion and limitation

A discussion is in order regarding the challenges and limitations of the statistical analysis of satellite reliability in general, and the analysis and results in this chapter in particular. First, note that the results provided here represent the “collective” failure behavior of Earth-orbiting satellites launched between 1990 and 2008. It can be argued, however, that no two satellites are truly alike, and that every satellite operates in a distinct environment, in different orbits, or even within the same orbit, where satellites, unless they are co-located, are exposed to different space environment conditions. The situation in the space industry is different from that for example in the semiconductor industry where data on, say, thousands of identical transistors operating under identical environmental conditions are available for statistical analysis, or in other industries with items for which failure data can be easily obtained from accelerated testing or field operation. The consequence is that in the absence of “satellite mass production,” statistical analysis of satellite failure and reliability data faces the dilemma of choosing between calculating precise “average” satellite reliability and deriving a possibly uncertain “specific” satellite platform reliability. This dilemma is explained in the following two possible approaches. The first approach is to lump together different satellites and analyze their “collective” on-orbit failure behavior, assuming that the failure times of the satellites are independent and identically distributed (iid). The advantage of doing so is that one can work with a relatively large sample (a few hundred or thousand units), as done in this chapter, and thus obtain some precision and a narrow confidence interval for the “collective” reliability analyzed (a single-digit percentage point dispersion). The disadvantage is that the iid assumption can be challenged, and the “collective” reliability calculated (with precision) may not reflect the specific reliability of a particular type of spacecraft in a particular orbit. The second approach is to specialize the data, for example, for a specific spacecraft platform or mission type, or for satellites in particular orbits. The advantage of doing so is that the reliability analyzed is specific to the type of spacecraft considered (it is no longer a “collective” on-orbit reliability). The disadvantage is that the sample size is reduced, and, as a consequence, the confidence interval expands. Given the available number of satellites (a few thousands), inappropriate data specialization, which could reduce the sample size to, say, fewer than a hundred data points, will result in significantly large confidence intervals, and thus highly dispersed and uncertain “specific” satellite reliability results. This chapter provided results based on the first approach, the “collective” failure behavior of satellites recently launched. The second approach is adopted in Chapter 4 and Appendix A where reliability results based on careful data specialization by satellite mission type, orbit type, and mass category are derived.



February 17, 2011

10:3



2.A

21

Appendix

Table 2.A.1 Tabular data for the Kaplan–Meier plot of satellite reliability in Figure 2.3. Failure time ti (years) 0.0027 0.0055 0.0082 0.0110 0.0137 0.0192 0.0246 0.0329 0.0411 0.0438 0.0630 0.0986 0.1396 0.1451 0.1752 0.1862 0.1999 0.2163 0.2437 0.2793 0.2930 0.3368 0.3504 0.3587 0.4572 0.5202 0.5394 0.6051 0.6270 0.6489

ˆ i) R(t

Failure time ti (years)

ˆ i) R(t

Failure time ti (years)

ˆ i) R(t

0.9975 0.9968 0.9956 0.9943 0.9930 0.9924 0.9918 0.9911 0.9898 0.9885 0.9879 0.9872 0.9865 0.9859 0.9852 0.9845 0.9838 0.9831 0.9824 0.9817 0.9810 0.9803 0.9796 0.9789 0.9782 0.9775 0.9768 0.9761 0.9754 0.9747

0.6899 0.7420 0.8460 0.8597 0.8679 0.9144 0.9966 1.2731 1.4100 1.9055 1.9192 1.9521 1.9767 1.9822 2.1547 2.8830 2.9377 3.0719 3.1376 3.1951 3.2416 3.3758 3.4387 3.6879 3.9918 4.2464 4.4819 4.8679 4.9199 5.0267

0.9740 0.9732 0.9725 0.9718 0.9711 0.9703 0.9696 0.9688 0.9681 0.9673 0.9665 0.9657 0.9649 0.9641 0.9633 0.9624 0.9616 0.9607 0.9598 0.9590 0.9581 0.9572 0.9564 0.9555 0.9545 0.9536 0.9527 0.9517 0.9507 0.9496

5.3854 5.5003 5.7248 5.7413 5.7440 5.9713 5.9986 6.1246 6.6502 6.6639 6.7680 7.0554 7.0637 7.1841 7.3977 7.9863 8.0684 8.1123 8.3176 8.4244 8.6489 8.9473 9.4593 10.0862 10.2916 11.4771 11.8385 13.4401 13.8070 14.2560

0.9486 0.9475 0.9464 0.9453 0.9442 0.9430 0.9419 0.9408 0.9396 0.9384 0.9372 0.9359 0.9347 0.9334 0.9322 0.9308 0.9295 0.9281 0.9267 0.9253 0.9238 0.9223 0.9207 0.9188 0.9169 0.9142 0.9113 0.9074 0.9031 0.8983



22

February 17, 2011

10:3



Table 2.A.2 Tabular data for the confidence intervals of satellite reliability in Figure 2.4. 95% confidence interval

95% confidence interval

95% confidence interval

Failure time (years)

Lower bound

Upper bound


Lower bound

Upper bound


Lower bound

Upper bound

0.0027 0.0055 0.0082 0.0110 0.0137 0.0192 0.0246 0.0329 0.0411 0.0438 0.0630 0.0986 0.1396 0.1451 0.1752 0.1862 0.1999 0.2163 0.2437 0.2793 0.2930 0.3368 0.3504 0.3587 0.4572 0.5202 0.5394 0.6051 0.6270 0.6489

0.9950 0.9941 0.9923 0.9906 0.9889 0.9881 0.9873 0.9865 0.9849 0.9833 0.9824 0.9816 0.9808 0.9800 0.9792 0.9783 0.9775 0.9767 0.9759 0.9750 0.9742 0.9734 0.9725 0.9717 0.9709 0.9701 0.9692 0.9684 0.9676 0.9667

0.9999 0.9996 0.9988 0.9980 0.9971 0.9967 0.9962 0.9958 0.9948 0.9938 0.9933 0.9928 0.9923 0.9917 0.9912 0.9907 0.9901 0.9896 0.9890 0.9884 0.9879 0.9873 0.9867 0.9862 0.9856 0.9850 0.9844 0.9838 0.9832 0.9826

0.6899 0.7420 0.8460 0.8597 0.8679 0.9144 0.9966 1.2731 1.4100 1.9055 1.9192 1.9521 1.9767 1.9822 2.1547 2.8830 2.9377 3.0719 3.1376 3.1951 3.2416 3.3758 3.4387 3.6879 3.9918 4.2464 4.4819 4.8679 4.9199 5.0267

0.9659 0.9650 0.9642 0.9634 0.9625 0.9617 0.9608 0.9600 0.9591 0.9582 0.9572 0.9563 0.9554 0.9545 0.9535 0.9525 0.9516 0.9506 0.9496 0.9486 0.9476 0.9466 0.9456 0.9445 0.9435 0.9424 0.9413 0.9402 0.9390 0.9378

0.9820 0.9814 0.9808 0.9802 0.9796 0.9790 0.9784 0.9777 0.9771 0.9764 0.9757 0.9751 0.9744 0.9737 0.9730 0.9723 0.9716 0.9708 0.9701 0.9694 0.9686 0.9679 0.9671 0.9664 0.9656 0.9648 0.9640 0.9631 0.9623 0.9614

5.3854 5.5003 5.7248 5.7413 5.7440 5.9713 5.9986 6.1246 6.6502 6.6639 6.7680 7.0554 7.0637 7.1841 7.3977 7.9863 8.0684 8.1123 8.3176 8.4244 8.6489 8.9473 9.4593 10.0862 10.2916 11.4771 11.8385 13.4401 13.8070 14.2560

0.9366 0.9353 0.9340 0.9328 0.9315 0.9302 0.9289 0.9276 0.9262 0.9248 0.9234 0.9220 0.9205 0.9191 0.9176 0.9161 0.9145 0.9129 0.9113 0.9096 0.9080 0.9062 0.9043 0.9020 0.8997 0.8963 0.8926 0.8873 0.8813 0.8747

0.9605 0.9596 0.9587 0.9578 0.9568 0.9559 0.9549 0.9540 0.9530 0.9520 0.9509 0.9499 0.9488 0.9478 0.9467 0.9456 0.9444 0.9433 0.9421 0.9409 0.9397 0.9384 0.9371 0.9356 0.9340 0.9321 0.9300 0.9275 0.9248 0.9219



February 17, 2011

7:37


3

Parametric analysis and Weibull modeling of spacecraft reliability1 Nonparametric analysis provides powerful results since the reliability calculation is unconstrained to fit any particular predefined lifetime distribution. However, this flexibility makes nonparametric results neither easy nor convenient to use for various purposes often encountered in engineering design (e.g., reliability-based design optimization). In addition, some failure trends and patterns are more clearly identified and recognizable with parametric analysis. Several methods are available to fit parametric distributions to the nonparametric reliability results (as provided for example by the Kaplan–Meier estimator), or to derive parametric reliability distributions directly from the failure and censored times. In the following, two such methods are briefly reviewed, the probability plots and the maximum likelihood estimation (MLE) method. The former is used to illustrate that satellite reliability can be properly approximated by a Weibull distribution, and the latter is used to calculate the parameters of the Weibull distribution. Following these calculations, a more advanced parametric fit is provided based on Weibull mixture distributions, and it will be shown that the resulting mixture distribution properly captures all the trends in the nonparametric spacecraft reliability results.

1 This chapter is based on three articles written by the authors and published in the Journal of Spacecraft

and Rockets (Castet and Saleh, 2009a) and in Reliability Engineering and System Safety (Castet and Saleh, 2009b, 2010a). Spacecraft Reliability and Multi-State Failures: A Statistical Approach, First Edition. Joseph Homer Saleh and Jean-François Castet. © 2011 John Wiley & Sons, Ltd. Published 2011 by John Wiley & Sons, Ltd. ISBN: 978-0-470-68791-8



February 17, 2011

7:37


24


3.1

Weibull distribution: an overview

The Weibull distribution is one of the most commonly used distributions in reliability analysis. The reason for its wide adoption is that it is quite flexible, and with an appropriate choice of one of its two parameters (the shape parameter, as discussed below), it can model different kinds of failure behaviors. More specifically, the Weibull distribution can model an increasing failure rate (wear-out), a decreasing failure rate (infant mortality), or a constant failure rate (i.e., an exponential distribution). The Weibull distribution has two parameters: the shape parameter β and the scale parameter θ . Its failure rate can be written as follows: λ(t) =

β θ

β−1 t θ

with θ > 0, β > 0, t ≥ 0.

(3.1)

The shape parameter β is dimensionless, and the scale parameter θ is expressed in units of time. Its probability density function can be expressed as follows: β f (t; β, θ ) = θ

β−1 t β t exp − . θ θ

(3.2)

The resulting reliability function is t β R(t) = exp − . θ

(3.3)

The flexibility of the Weibull distribution is captured by the effect of the Weibull shape parameter β on the failure rate. More specifically: 1. For 0 < β < 1, the failure rate is decreasing, thus a choice of β within this range allows infant mortality to be modeled. 2. For β = 1, the failure rate is constant, and the Weibull distribution in this case is equivalent to the exponential distribution. 3. For β > 1, the failure rate is increasing, thus a choice of β greater than one allows wear-out failures to be modeled. More specifically: (a) for 1 < β < 2, an increasing concave failure rate is observed; (b) for β = 2, a linear failure rate is observed, and the Weibull distribution in this case is equivalent to the Rayleigh distribution; (c) for β > 2, an increasing convex failure rate is observed; (d) for 3 ≤ β ≤ 4, the Weibull distribution approaches the normal distribution.



February 17, 2011

7:37


PARAMETRIC ANALYSIS AND WEIBULL MODELING OF RELIABILITY

3.2 3.2.1

25

Probability plots or graphical estimation Procedure

Probability plots constitute a simple and visually appealing graphical estimation procedure for fitting a parametric distribution to nonparametric data. This procedure is based on the fact that some parametric models such as the exponential or Weibull distribution can have their reliability function linearized using a particular mathematical transformation. Consider for example the Weibull distribution. By taking the natural logarithm of both sides of (3.3), the following result is obtained: ln [R(t)] = −

β t . θ

(3.4)

Taking again the natural logarithm of the (negative of the) two sides of this equality, ln [− ln R(t)] = β ln(t) − β ln(θ )

(3.5)

Equation (3.5) is equivalent to (3.3): t β R(t) = exp − θ

⇔

ln [− ln R(t)] = β ln(t) − β ln(θ ).

(3.6)

ˆ i ), is Given the equivalence in (3.6), if nonparametric reliability data for an item, R(t provided at different points in time ti , but the underlying parametric distribution is not known, it can be easily tested whether a Weibull distribution is an appropriate fit for the data, or not, through the following “graphical estimation” technique:

r If ln[− ln R(t ˆ i )] as a function of ln(ti ) is plotted and data points are aligned

ˆ in the (ln(t); ln[− ln R(t)]) space (the resulting graph is termed the Weibull plot), then it can be concluded that the data effectively arises from a Weibull distribution (i.e., the underlying parametric distribution is indeed a Weibull).

r If the data points are not aligned, then the Weibull distribution can be dismissed as an unfit parametric model for the data (this follows from the equivalence in (3.6)).

3.2.2

Weibull plot and Weibull fit of spacecraft reliability

Figure 3.1 shows the Weibull plot for the nonparametric satellite reliability. The data points are well aligned, and a regression analysis provides the following result: y = 0.3875x − 3.4972 with R 2 = 0.9835.

(3.7)



26

February 17, 2011

7:37


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES -1 Empirical data Linear least squares fit

-2

ln[–ln R(t)]

-3 -4 -5 -6 -7 -7

-6

-5

-4

-3

-2 -1 ln(t)

0

1

2

3

4

Figure 3.1 Weibull plot of satellite reliability.

This provides a first indication that the Weibull fit is indeed a good one, and that satellite reliability can be justifiably approximated by a Weibull distribution. In addition, the slope of the least squares fit shown in Figure 3.1 and captured in (3.7) provides the shape parameter β of the Weibull distribution. The scale parameter θ can be evaluated for example from the value of the intersection of the line with the y-axis. The parameters derived from this graphical estimation method are as follows: β = 0.3875 θ = 8316 (years).

(3.8)

The resulting Weibull distribution for satellite reliability is given by replacing these parameters in (3.3). The result is shown in Figure 3.2. The goodness of fit of this distribution is discussed later in this work.

3.2.3

Advantages and limitations of the graphical estimation method

Probability plots or graphical estimation methods for parametric fit have a powerful advantage in their simplicity: they are easy to set up, they do not require involved calculations, and they provide immediate visual information about the validity of the assumed parametric distribution. In addition, the parameters of the assumed distribution, in the case of this work the Weibull distribution, can be calculated by a simple linear least squares fit of the data on probability plots. However, probability


February 17, 2011

7:37



27

1.00 0.99 0.98

Nonparametric estimation Weibull fit

0.97 Reliability


0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0


Figure 3.2 Kaplan–Meier estimated satellite reliability and (the graphical) Weibull fit. plots have some disadvantages when used to calculate the actual parameters of the distribution. For example: 1. With distributions requiring logarithmic time transformations, such as the Weibull distribution, excessive weight is given to early failure times and not enough weight to later failure times. As a consequence, the resulting parametric fit is biased (toward more precision for early failure times). 2. As a consequence of comment 1, the least squares fit on the probability plot does not result in a minimum variance estimate of the actual distribution. 3. The estimation of the parameters may be poor if the failure times are not scattered properly across the data range. If the objectives of a reliability study do not require overall “precise” results, then probability plots or graphical estimations are adequate for conducting parametric fits. Otherwise, one should revert to the more precise maximum likelihood estimation method discussed next.

3.3

Maximum likelihood estimation (MLE)

The MLE method addresses the limitations of probability plots and provides more precise parametric fits than graphical estimation, as long as the sample size is not exceedingly small (e.g., in the single digits). The MLE method is analytically more involved than the graphical estimation techniques, and requires: (1) determining of



28

February 17, 2011

7:37



the right formulation of the likelihood function for a chosen distribution and type of censoring, as will be shown shortly; and (2) searching for an optimum of this function, which can be accomplished through various computational or analytical techniques. The values of the unknown parameters of the distribution that maximize the likelihood function are termed the maximum likelihood estimates and the method is known as the MLE method. The analytical derivation of the MLE in the case of a Weibull distribution is provided in the next subsection.

3.3.1

MLE derivation of a Weibull distribution

The general derivation of the MLE of a Weibull distribution is provided in this subsection and based on a presentation of the method in Lawless (2003). The reader interested in the results of the MLE for spacecraft reliability can skip this subsection and jump ahead to Section 3.3.2. Conceptually, the MLE is based on the following idea: given a set of observed data, and assuming a parametric life distribution with unknown parameters represented by the vector θ (e.g., two parameters for the Weibull distribution), a likelihood function is defined as the probability of obtaining or generating the observed data from the chosen parametric distribution. When the probability density function has a parametric expression f (t; θ ), and iid lifetimes t1 , t2 , . . . , tn are collected for a sample of n items, the likelihood function can be written as follows: L(θ) =

n

f (ti ; θ ).

(3.9)

i=1

Next, a search for a maximum of L(θ ) is conducted to obtain an estimate θˆ of θ and an estimate of the reliability function R(t; θˆ ) according to the hypothesized probability distribution. Difficulty arises when the data is censored, and (3.9) must be modified to take the censoring mechanism into account. To this extent, consider n items having the lifetimes t1 , t2 , . . . , tn . Let the variable δ i account for the censoring in the data: if ti is an observed lifetime, δ i = 1; if ti is a censoring time, δ i = 0. Then, for the censoring type encountered in this study, the likelihood function can be written as follows, assuming the lifetimes and censoring times are independent: L(θ) =

n

f (ti ; θ )δi R(ti ; θ )1−δi

(3.10)

i=1

where f (t; θ ) and R(t; θ) are respectively the probability density function (p.d.f.) and the reliability function of the hypothesized probability distribution. The likelihood function needs to be customized, that is, derived specifically, for each parametric distribution under consideration to be fitted to the data, since, as seen in (3.9) and (3.10), f (t; θ ) determines the likelihood function. For the Weibull



February 17, 2011

7:37



29

distribution, a more convenient alternative to (3.2) and (3.3) uses the equivalent extreme value distribution for Y = ln T, with the following p.d.f. and reliability function: 1 (y−u)/b e exp −e(y−u)/b , −∞ < y < ∞ b R(y; u, b) = exp −e(y−u)/b , −∞ < y < ∞ f (y; u, b) =

(3.11) (3.12)

where y = ln t, u = ln θ , and b = β −1 . Since the maximization process is unchanged by a monotonic mathematical transformation, it is usually more convenient to use the log-likelihood function l(θ ) = ln L(θ ). By using the change of variables z = (y – u)/b, and introducing r = δ i , the log-likelihood function is expressed as n

δi z i − e zi . l(u, b) = −r ln b +

(3.13)

i=1

It is now possible to maximize the log-likelihood function l(u, b), or equivalently minimize –l(u, b). The minimization process is rendered more efficient by computing the gradient and the Hessian of –l(u, b): ⎛

⎞

⎛

1 δi − e zi b i=1 n

⎞

∂l ⎜ ⎟ ⎟ ⎜ − ∂u ⎟ ⎜ ⎜ ⎟ ⎟=⎜ − ∇l(u, b) = ⎜ ⎟ ⎝ ∂l ⎠ ⎜ n ⎟

r 1 ⎝ zi ⎠ − δ z − e + i i ∂b b b i=1

(3.14)

⎡

⎤ ∂ 2l ∂ 2l − − ⎢ ∂u 2 ∂u∂b ⎥ ⎥ −H (u, b) = ⎢ ⎣ ⎦ ∂ 2l ∂ 2l − 2 − ∂u∂b ∂b

(3.15)

where −

n ∂ 2l 1 zi = e ∂u 2 b2 i=1

n ∂ 2l 1 zi − = 2 e − δi + z i e zi ∂u∂b b i=1 n

zi 1 ∂ 2l 2 zi 2z i e − δi + z i e . − 2 = 2 −r + ∂b b i=1

(3.16) (3.17)

(3.18)



30

February 17, 2011

7:37



Table 3.1 Weibull parameters for satellite reliability derived using two methods. Method Graphical estimation MLE

3.3.2

Shape parameter β

Scale parameter θ (years)

0.3875 0.4521

8316 2607

MLE Weibull fit for spacecraft reliability

The MLE procedure outlined in the previous subsection is applied to the failure and censored data of the sample considered here, namely, 1584 Earth-orbiting satellites successfully launched between 1990 and 2008. The results of the MLE Weibull parameters of spacecraft reliability are given in Table 3.1, and are shown alongside the previous results derived using the graphical estimation method. Recall that the Weibull reliability function in this work is written as follows: t β R(t) = exp − . θ

(3.19)

The shape and scale parameters from Table 3.1 can be inserted into (3.21) to obtain the spacecraft parametric reliability models. The Weibull distributions derived using the two different statistical estimation techniques, the graphical and MLE methods, are shown in Figure 3.3, along with the nonparametric results derived in the previous chapter, hereafter referred to as the “benchmark reliability.” The quality of each fit of each distribution is discussed in the next section. 1.00 0.99 0.98

Nonparametric estimation Weibull fit (graphical) Weibull fit (MLE )

Reliability

0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0


Figure 3.3 Satellite reliability and Weibull fits.



February 17, 2011

7:37



31

Before proceeding, it is worth noting that both estimation methods provide shape parameters of the Weibull distributions (β) smaller than one, which indicates that satellite infant mortality is a robust finding.

3.4 3.4.1

Comparative analysis of the spacecraft reliability parametric fits MLE versus graphical estimation Weibull fits

Both estimation methods provide relatively precise approximations of the “benchmark reliability” results as can been seen from Figure 3.3. The error between the “benchmark reliability” and the parametric approximation remains smaller than 1.12 percentage point for the MLE fit and less than 2.05 percentage point for the graphical estimation fit. In addition, the average error between the “benchmark reliability” and the parametric approximation is 0.42 percentage point for the MLE fit and 0.71 percentage point for the graphical estimation fit (Table 3.2). These results and Figure 3.3 clearly indicate that the MLE fit is a more precise fit of satellite reliability than the graphically estimated Weibull parameters. It is worth taking a closer look at its error dynamics as a function of time. Notice for example in Figure 3.3 that the quality of the graphical fit starts degrading roughly after 7 years on orbit (as evidenced by the growing gap between the graphical Weibull fit and the nonparametric result) and, at t = 15 years, the parametric result overestimates the actual satellite reliability by roughly 2 percentage point. The MLE fit, however, “sustains” its precision throughout the time horizon, and, at t = 15 years, the MLE parametric result overestimates the actual satellite reliability by less than 1 percentage point. The degradation of the graphical fit can be traced back to the Weibull plot (Figure 3.1) and the comment 1 in Section 3.2.3. Notice that the least squares fit undershoots the empirical data in a small region in the upper right corner of Figure 3.1 starting around ln(t) = 2. This value in effect corresponds to t = 7.4 years, which is roughly where the graphical fit starts degrading (as seen in Figure 3.3). The least squares fit has a smaller slope than the local slope of the data points after ln(t) = 2, which implies that its fitted shape parameter (as given by the slope of the line) is also smaller, hence more optimistic than the actual local shape parameter. As a result, the fitted reliability overestimates the actual reliability after t = 7 years. Incidentally, the time horizon for this study, t = 15 years, corresponds to ln(t) = 2.7. The fact that the quality of the graphical fit steadily degrades from 7 to 15 years is due to this minor mismatch in a small region on the Weibull plot where 2 < ln(t) < 2.7. This constitutes an intrinsic weakness of the graphical estimation method and results from the logarithmic time transformation required for the Weibull plot. However, this same weakness provides a useful feature of the graphical Weibull fit: that is, it is more precise than the MLE in capturing infant mortality and approximating the actual satellite reliability up to t = 7 years. In other words, while the MLE fit “sustains” its precision throughout the time range in this study, the graphical fit has



32

February 17, 2011

7:37



Table 3.2 Error between the nonparametric and parametric satellite reliability over 7 and 15 years. Over 7 years Maximum error Method

Over 15 years

Average error

(percentage point)

Graphical MLE

0.79 0.94

0.37 0.51

Maximum error

Average error

(percentage point) 2.05 1.12

0.71 0.42

two “regimes”: it is more precise than the MLE in the early years up to t = 7 years; and it is less precise than the MLE for later years (t > 7 years). The average and maximum errors within these two time periods are given in Table 3.2. A more detailed analysis of the errors (residuals) of both parametric fits with respect to the benchmark reliability is provided in Figure 3.4. Two box-plots for the residuals of the MLE and graphically estimated Weibull fits are shown in Figure 3.4. A box-plot reads as follows: the lower boundary of the “box” is determined by the first quartile (25th percentile) of the residuals, and the upper boundary by the third quartile (75th percentile). The line within the box indicates the median of the residuals. The “whiskers” outside the box represent the minimum and maximum of the residuals. Figure 3.4 confirms the higher precision of the MLE fit over 15 years. The residuals of the MLE have a smaller spread than those of the graphical fit: (1) the 25th and 75th percentiles are less dispersed for the MLE (i.e. smaller box); and (2) the extreme values are less spread (i.e., shorter whiskers).

Residual (percentage point)

2.0 1.5 1.0 0.5 0.0 -0.5 -1.0 -1.5 -2.0 Graphical estimation

MLE

Figure 3.4 Box-plots of the residuals between the Weibull fits and the nonparametric reliability over 15 years.


February 17, 2011

7:37



33

2.0 Residual (percentage point)


1.5 1.0 0.5 0.0 -0.5 -1.0 -1.5 -2.0 Lognormal - MLE

Weibull - MLE

Figure 3.5 Residuals of the lognormal versus Weibull MLE fits.

3.4.2

MLE Weibull versus MLE lognormal fits

In addition to the Weibull distribution for parametrically model spacecraft reliability, an MLE lognormal fit was also conducted, and the resulting p.d.f. is f (t; μ, σ ) =

1 2 2 √ e−(ln(t)−μ) /(2σ ) tσ 2π

(3.20)

with μ = 9.7646 years and σ = 5.2209 years. The residuals of both distributions, the Weibull and the lognormal, are shown in Figure 3.5, and they indicate that although both distributions are relatively accurate representations of the nonparametric (benchmark) satellite reliability results, the lognormal distribution is less precise and a more biased fit of satellite reliability than the Weibull distribution.

3.5

Finite mixture distributions

As noted previously, several trends are present in the nonparametric result of spacecraft reliability. Weibull distributions, which have monotonic failure rates, can only capture one of the satellite failure trends observed, but fail to model the others. The same observations can be made regarding the inability of the lognormal distribution to accurately model the nonparametric satellite reliability. This discussion indicates the need of finite mixture distributions to accurately represent the nonparametric spacecraft reliability. Unlike the Weibull distribution, mixtures of distributions can model nonmonotonic failure rates and are further discussed next. Several distributions (e.g., exponential, Weibull, lognormal) can be used as a basis for linear combination to generate a mixture. In this section, to build on the Weibull fit presented in the previous section, it was decided to model the nonparametric satellite



34

February 17, 2011

7:37



reliability with a mixture of Weibull distributions. In the following, the parameters of the Weibull mixture distribution are determined using different methods, and the goodness of fit of the mixture distribution is compared to the previous single Weibull distribution model. The parametric reliability model with a mixture of Weibull distributions can be expressed as follows: t βj α j exp − R(t) = θj j=1 k

(3.21)

where k is the number of distributions in the mixture 0 ≤ αj ≤ 1 k

αj = 1

j=1

A mixture distribution with two Weibull distributions is considered next, and different methods are presented to compute the value of the different parameters of the mixture.

3.5.1

Methods for estimating parameters of mixture distributions

Several methods are available to estimate the different parameters of the mixture distributions, such as the nonlinear least squares method on the nonparametric reliability, or the maximum likelihood estimation (directly using the lifetimes and censored data). A list of other methods and their details can be found in Titterington et al. (1985). 3.5.1.1

Nonlinear least squares method

The nonlinear least squares method is a computationally “brute force” method that seeks to minimize the sum of the squares of the differences between the nonparametric reliability and the mixture distribution by searching over the values of the parameters of the mixture, while satisfying the constraints on α j , j ∈ [1, k]. 3.5.1.2

Maximum likelihood estimation

The conceptual approach of this method has been described in Section 3.3. The expression of the likelihood function in terms of the p.d.f. and reliability function remains the same. However, the expressions for the p.d.f and reliability function are more complex in the case of a mixture compared to the single-distribution case



February 17, 2011

7:37



35

described in Section 3.3. Recall that we are considering n items having the lifetimes t1 , t2 , . . . , ti , . . . , tn , i ∈ [1, n]. As previously, it is more convenient to work with the extreme value distribution than the Weibull distribution: 1 (yi −u j )/b j e exp −e(yi −u j )/b j bj R j (yi , u j , b j ) = exp −e(yi −u j )/b j j ∈ [1, k] f j (yi , u j , b j ) =

(3.22)

where yi = ln(ti ) u j = ln(θ j ) b j = 1/β j . Using (3.22), the expressions for the p.d.f. and reliability function are (with k = 2) f (yi , u 1 , b1 , u 2 , b2 , α) = α · f 1 (yi , u 1 , b1 ) + (1 − α) · f 2 (yi , u 2 , b2 ).

(3.23)

R(yi , u 1 , b1 , u 2 , b2 , α) = α · R1 (yi , u 1 , b1 ) + (1 − α) · R2 (yi , u 2 , b2 ).

(3.24)

Let the parameter vector θ encompass the five parameters of the mixture θ = [u 1 , b1 , u 2 , b2 , α]T .

(3.25)

Hence, using (3.10), the likelihood function can be expressed as L(θ) =

n

f (yi ; θ )δi R(yi ; θ )1−δi .

(3.26)

i=1

The log-likelihood function l(θ ) = ln L(θ ) can be written as l(θ) =

n

[δi ln f (yi , θ ) + (1 − δi ) ln R(yi , θ )].

(3.27)

i=1

As previously, it is now possible to maximize the log-likelihood function l(θ), or equivalently minimize –l(θ). Several methods are available to obtain the maximum likelihood estimate θˆ of θ , and two of them are listed below (additional ones can be found in (Titterington et al., 1985)): 1. Expectation–maximization (EM) algorithm method. 2. Classic optimization problem on l(θ) using for example the Newton–Raphson method. The latter was chosen to solve this problem with a gradient-based algorithm, and the expression for the gradient of the (negative) log-likelihood function –l(θ) is provided



36

February 17, 2011

7:37



in the following. The reader interested in the EM method is referred to the seminal work by Dempster et al. (1977), or to McLachlan and Krishnan (2008), Titterington et al. (1985), and Kvam and Vidakovic (2007). Thus, ∂l ∂l ∂l ∂l T ∂l ,− ,− ,− ,− − ∇l(θ ) = − ∂u 1 ∂b1 ∂u 2 ∂b2 ∂α

(3.28)

where, by using the change of variables z 1i = (yi − u 1 ) b1 and z 2i = (yi − u 2 ) b2 , i ∈ [1, n], −

n

α R1 (yi , u 1 , b1 ) ∂l f 1 (yi , u 1 , b1 ) α =− δi (e z1i − 1) + (1 − δi ) e z1i ∂u 1 b1 f (yi , θ) b1 R(yi , θ) i=1

−

n

∂l f 1 (yi , u 1 , b1 ) α α R1 (yi , u 1 , b1 ) δi (z 1i e z1i − z 1i − 1) + (1 − δi ) z 1i e z1i =− ∂b1 b1 f (yi , θ) b1 R(yi , θ) i=1

(3.29)

(3.30) −

n

1 − α z2 ∂l f 2 (yi , u 2 , b2 ) 1 − α z2 R2 (yi , u 2 , b2 ) + (1 − δi ) =− (e i − 1) e i δi ∂u 2 b2 f (yi , θ) b2 R(yi , θ) i=1

−

n

∂l f 2 (yi , u 2 , b2 ) 1−α 1−α R2 (yi , u 2 , b2 ) δi + (1 − δi ) =− (z 2i e z2i − z 2i − 1) z 2i e z2i ∂b2 b2 f (yi , θ) b2 R(yi , θ) i=1

(3.31)

(3.32) −

n

f 1 (yi , u 1 , b1 ) − f 2 (yi , u 2 , b2 ) ∂l R1 (yi , u 1 , b1 ) − R2 (yi , u 2 , b2 ) =− + (1 − δi ) . δi ∂α f (yi , θ) R(yi , θ) i=1

(3.33)

3.5.2

The 2-Weibull mixture distribution of spacecraft reliability

The resulting parameters of the 2-Weibull mixture distribution with the MLE method are presented in Table 3.3. Table 3.3 Parameters of the 2-Weibull mixture distribution with the MLE method. Method MLE

β1

θ 1 (years)

β2

θ 2 (years)

α

0.3760

14310.1

2.9937

9.3

0.9725


February 17, 2011

7:37


PARAMETRIC ANALYSIS AND WEIBULL MODELING OF RELIABILITY 1.00

37

Nonparametric estimation 2-Weibull mixture (MLE)

0.99 0.98 0.97 Reliability


0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure 3.6 The 2-Weibull mixture reliability model and nonparametric spacecraft reliability. The resulting spacecraft reliability model is therefore written as follows: R(t) = 0.9725 exp −

t 14310.1

0.3760

t 2.9937 + 0.0275 exp − . 9.3

(3.34)

Note that the first Weibull shape parameter β 1 < 1 captures spacecraft infant mortality, whereas the second Weibull shape parameter β 2 > 1 captures spacecraft wear-out failures. Figure 3.6 shows the result of this Weibull mixture distribution model superimposed on the nonparametric spacecraft reliability results. Some measures of the goodness of fit of the 2-Weibull mixture distribution are presented in Table 3.4. And as can be seen on the normal probability plot in Figure 3.11 below, the residuals between the mixture model and the nonparametric reliability results are quasi-normally distributed, which is a good indication that no bias remains in the parametric model and all failure trends have been captured by the mixture. Table 3.4 Goodness of fit measures for the 2-Weibull mixture distribution. Residuals Average error Maximum error 25th percentile 75th percentile Method MLE

R2 0.9930

(percentage point) 0.17

0.57

−0.14

0.16



February 17, 2011

7:37


38


3.6

Comparative analysis of the single versus the mixture distribution Weibull fits

In this section, the quality of the fit between a single Weibull distribution and a 2-Weibull mixture distribution are compared and assessed with respect to the nonparametric spacecraft reliability. The single Weibull distribution was derived in Section 3.3.2 with the MLE procedure. The resulting spacecraft reliability model is given by 0.4521 t R(t) = exp − . 2607

(3.35)

For the 2-Weibull mixture distribution, also derived using an MLE procedure, the resulting spacecraft reliability model is given by R(t) = 0.9725 exp −

t 14310.1

0.3760

t 2.9937 + 0.0275 exp − . 9.3

(3.36)

These two parametric models of the spacecraft reliability are shown in Figure 3.7 and Figure 3.8, superimposed on the nonparametric reliability results derived in the previous chapter. It can be observed that both parametric models provide relatively precise approximations of the nonparametric reliability as in Figure 3.7 and Figure 3.8. However, upon closer inspection, it is clear that the 2-Weibull mixture distribution follows with a higher accuracy the trends present in the nonparametric spacecraft reliability. To quantitatively gauge the accuracy improvement between the single Weibull and 1.00 0.99 0.98

Nonparametric estimation Single Weibull distribution (MLE)

Reliability

0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure 3.7 Nonparametric and single Weibull reliability.


February 17, 2011

7:37



39

1.00 Nonparametric estimation 2-Weibull mixture (MLE)

0.99 0.98 0.97 Reliability


0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89


Figure 3.8 Nonparametric and 2-Weibull mixture reliability. the 2-Weibull mixture distributions, the maximum and average errors are calculated between the nonparametric benchmark reliability results and the parametric models. The results are shown in Table 3.5 and provide a rough first measure of the accuracy improvement between the two parametric models. Table 3.5 shows that both parametric models are quite accurate with a maximum error of roughly 1.1 percentage point for the single Weibull and 0.6 percentage point for the 2-Weibull mixture distribution. In other words, by using for example (3.35) to model spacecraft reliability, the result will be at most 1.1 percentage point from the actual nonparametric reliability results. Figure 3.7 and Figure 3.8 show additional information regarding the quality of fit of both models, which is not captured by the simple statistics in Table 3.5. For example, it can be seen in Figure 3.8 that the 2-Weibull mixture distribution sustains a high precision through the 15-year time window, whereas the precision of the MLE fit varies as seen in Figure 3.7. More specifically, the MLE fit is precise in the first year, then its accuracy degrades between 1 and 7 years. This comment has to be put into perspective: the accuracy of the single Weibull model still remains within 1 percentage point of the nonparametric Table 3.5 Error between the nonparametric reliability and the parametric models over 15 years. Maximum error Parametric fit Single Weibull 2-Weibull mixture

Average error


0.42 0.17



40

February 17, 2011

7:37



reliability – to finish with a MLE parametric result overestimating the actual satellite reliability by less than a percentage point. Although both parametric models are adequate, the 2-Weibull mixture distribution is significantly more accurate than the single Weibull distribution in capturing the actual nonparametric spacecraft reliability. The average error for the 2-Weibull mixture distribution is 0.17 percentage point, which constitutes a remarkable parametric accuracy and a 60% improvement on the accuracy of the single Weibull fit. However, if high accuracy is not sought in a parametric model, then the single Weibull distribution with its simplicity (only two parameters) is more convenient to use and adopt than the five-parameter 2-Weibull mixture distribution. In addition to the calculation of the maximum and average errors, a more detailed analysis of the residuals of both parametric models is conducted with respect to the nonparametric reliability, as shown in Figure 3.9. This figure presents two box-plots for the residuals of the single Weibull and the 2-Weibull mixture distributions. Recall that the box-plot reads as follows: the lower boundary of the “box” is determined by the first quartile (25th percentile) of the residuals, and the upper boundary by the third quartile (75th percentile). The line within the box corresponds to the median value, and the “whiskers” outside the box represent the minimum and maximum of the residuals. Figure 3.9 confirms the higher precision of the mixture of distributions over 15 years. The residuals of the mixture distribution have a smaller spread than those of the single-function parametric fit: 1. The 25th and 75th percentiles are less dispersed for the mixture distribution (i.e., smaller box), as shown in Table 3.6.


1.0

0.5

0.0

-0.5

-1.0 Single Weibul - MLE

2-Weibull mixture - MLE

Figure 3.9 Box-plots of the residuals between the Weibull fits and the nonparametric reliability over 15 years.


February 17, 2011

7:37



41

Table 3.6 The 25th and 75th percentiles for the residuals of the parametric fits. 25th percentile Parametric fit

75th percentile

(percentage point) −0.28 −0.14

Single Weibull 2-Weibull mixture

0.51 0.16

2. The extreme values are less spread (i.e., shorter whiskers). The actual values are in part provided in Table 3.5. 3. The residuals of the 2-Weibull mixture distribution are clearly more symmetrically dispersed that those of the single Weibull (Figure 3.9). In addition, as noted earlier, the residuals between the 2-Weibull and the nonparametric reliability results are quasi-normally distributed, as shown Figure 3.11, which is a good indication that no bias remains in the parametric mixture model and all failure trends have been captured by the 2-Weibull mixture distribution (unlike the case of the single Weibull model, as shown in Figure 3.10).

0.999 0.997 0.99 0.98 0.95 0.90 Probability


0.75 0.50 0.25 0.10 0.05 0.02 0.01 0.003 0.001

-1

-0.5

0

0.5


Figure 3.10 Normal probability plot of the residuals of the single Weibull distribution.



42

February 17, 2011

7:37



Probability

0.999 0.997 0.99 0.98 0.95 0.90 0.75 0.50 0.25 0.10 0.05 0.02 0.01 0.003 0.001

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5


Figure 3.11 Normal probability plot of the residuals of the 2-Weibull mixture distribution. This last comment also indicates that it is superfluous to fit higher order mixture distributions (k > 2). As a conclusion to this chapter, the following suggestions are made for researchers and industry professionals, should they wish to use these spacecraft reliability results. First, the use of the nonparametric results is recommended as the most accurate reflection of actual spacecraft reliability. However, if the context of the study is not amenable to manipulating or using nonparametric results, then the use of the 2-Weibull mixture fit is recommended. The single Weibull MLE fit can be used if simplicity is sought and the study does not require a high level of precision.



February 18, 2011

7:31


4

Data specialization: statistical analysis of spacecraft reliability by orbit and mass categories1 4.1

Introduction

In the previous chapters, the “collective” failure behavior of Earth-orbiting satellites was analyzed, and parametric reliability models and nonparametric results were derived. Two important highlights from these analyses are that spacecraft exhibit infant mortality and their failure times are remarkably Weibull distributed. One limitation of the previous analysis, which was noted in Section 2.5, is that spacecraft of different types and in different orbits have been lumped together, and their collective or averaged failure behavior was statistically analyzed. The issue with such an approach is that the assumption that the failure times are iid may be challenged, and the collective reliability results may not accurately reflect the specific reliability of a particular spacecraft “type.” Given the relatively small number of spacecraft launched (a few thousand) and the significantly smaller number of on-orbit failures – spacecraft have been fairly reliable, as seen in the previous chapters – too narrow a data specialization, for example, for specific spacecraft platforms, significantly reduces the available sample size, thus constraining or preventing their 1 This

chapter was written in part in collaboration with Gregory F. Dubos, Thomas Hiriart, and Jarret M. Lafleur. It is based partly on two articles: one published in Acta Astronautica (Dubos et al., 2010); and the other presented at the 2009 International Astronautical Congress (Hiriart et al., 2009). Spacecraft Reliability and Multi-State Failures: A Statistical Approach, First Edition. Joseph Homer Saleh and Jean-François Castet. © 2011 John Wiley & Sons, Ltd. Published 2011 by John Wiley & Sons, Ltd. ISBN: 978-0-470-68791-8



44

February 18, 2011

7:31



statistical reliability analysis. In this chapter, a careful data specialization is devised for which the sample size and failure occurrence within the sample remain appropriate for statistical analysis, and the results of which are more specific than the previous collective or averaged spacecraft failure behavior. More specifically, this chapter specializes the data analysis by spacecraft mass category first, and then by mass and orbit. Nonparametric and parametric reliability results are derived for these different “types” of spacecraft. From a statistical or actuarial perspective, several parameters or characteristics of the spacecraft, such as its complexity, number of instruments, or power, to name a few, can potentially affect or be correlated with the probability of failure of the spacecraft. Since such parameters may also be associated with variations in spacecraft size, and because “mass” is a major consideration for the space industry, the first part of this chapter explores the correlation between spacecraft mass and reliability, and investigates whether different classes of spacecraft, defined in terms of mass, exhibit different reliability profiles. The following questions are addressed: Are different spacecraft masses correlated with different on-orbit failure behaviors? Do small spacecraft exhibit different on-orbit failure behaviors, hence different reliability profiles, than larger ones? And more broadly, do different classes of spacecraft (in terms of mass) have different reliability profiles? This possible correlation between spacecraft mass and reliability has been recognized by the industry, but to date it has not been statistically investigated. Intuitive trends have been discussed, but they often yielded contradictory conclusions. On the one hand, an increase in mass has naturally been associated with the use of design redundancy on board a spacecraft to improve reliability (Hecht, 1999). On the other hand, an increase in complexity (that can result in an increase in mass) has long been seen as a factor degrading reliability. This phenomenon of increased complexity and decreased reliability was identified during the Apollo program; for example, the pressure-fed and storable propellants on the lunar module propulsion systems allowed the use of ignition systems and pumps to be bypassed, resulting in a reduction of mass and complexity, and ultimately an increase in reliability (Williamson, 2006). Furthermore, recent work by Bearden (2003) indicated that NASA spacecraft that failed were more likely characterized by a high complexity factor (defined by the author as an average of technical factors including spacecraft mass) than a lower one. In addition, Fleeter (1999) discussed a simple model of spacecraft reliability R = R0n , where n is the number of components that “nominally scales with mass” and R0 is the reliability of each component. Using this relation, the author argues that “using the same part quality, [a small] spacecraft will be more reliable” than a larger one, as n will be smaller. As noted by Sarsfield (1998), the question of system size in relation to reliability divides the spacecraft community into, on one side, the proponents of small, “single-string,” and thus simple systems, and, on the other, the advocates of larger systems using more redundancy. Quantitative data-driven answers have therefore to be found to resolve this issue and identify the correlation, if any, between spacecraft mass and reliability. To this end, a statistical reliability analysis of satellites arranged by mass categories is first conducted in this chapter to investigate whether these different classes of spacecraft



February 18, 2011

7:31


DATA SPECIALIZATION: ANALYSIS BY ORBIT AND MASS

45

exhibit different failure behaviors and reliability profiles. Note that this analysis constitutes a phenomenological study of the reliability of spacecraft organized by mass categories, and that it does not assume that mass per se is a “cause” of on-orbit failures. The remainder of this chapter is organized as follows. In Section 4.2, the classification of spacecraft by mass is introduced. In Section 4.3, a nonparametric analysis of spacecraft reliability for each class is conducted using the Kaplan–Meier estimator, given the censored nature of the data, and different reliability profiles are identified for each spacecraft mass category. Parametric fits are then provided in Section 4.4, using the MLE procedure for single Weibull and 2-Weibull mixture distributions. Section 4.5 introduces and discusses an additional dimension for data specialization, namely, spacecraft orbits. Sections 4.6 and 4.7 present nonparametric and parametric analyses based on the new mass–orbit categories of spacecraft. Finally, beyond the reliability results obtained, hypotheses are proposed in Section 4.8 as possible causal explanations of the reliability and failure differences of the different mass–orbit spacecraft categories.

4.2

Data description and mass categorization

The same database and data restriction noted in Chapter 2 were adopted in this chapter. As in previous chapters, the reliability calculations were conducted using what is referred to in the database as a Class I failure, that is, the retirement of a satellite due to failure resulting in a total loss of functionality of the satellite. For each spacecraft in the sample, the following information was collected: (1) the spacecraft initial mass; (2) the launch date; (3) the failure date, if failure occurred; (4) the design lifetime; and (5) the “censored time,” if no failure occurred. The data collection template and sample data for the analysis are shown in Table 4.1. After the data collection, the satellites were categorized into different mass bins. Various taxonomies for spacecraft based on their mass have been used over time in the Table 4.1 Data collection template and sample data for the statistical analysis of satellite reliability. Sample unit number

Mass at launch (kg)

Satellite 1

1500

Satellite 2

480

.. . Satellite 1394

2600

Launch date Nov. 6, 2005 Mar. 1, 2002 .. . Apr. 26, 2005

Failure date (if failure occurred)

Design lifetime (years)

Censored time (if no failure occurred)

—

5

Oct. 2, 2008

—

3

Mar. 1, 2005

15

.. . —

.. . Mar. 28, 2006



46

February 18, 2011

7:31



Table 4.2 Categories of spacecraft based on mass. Mass at launch (kg)

Spacecraft category

Examples

[0–500] ]500–2500] >2500

Small (S) Medium (M) Large (L)

FAST, JASON 1, NANOSAT 01 TOPEX-POSEIDON, GPS NAVSTAR II-06 DirecTV 1R, HotBird 8

space industry. For example, Sarsfield (1998) points out that “there is no official definition of a small satellite,” even though this qualifier has gained much popularity during the last two decades. The author adds, “the Center for Satellite Engineering Research at the University of Surrey defines a ‘mini’ satellite as being between 100 and 500 kg.” Similarly, a list of satellites launched from 1991 to 1995, whose mass is under 425 kg and considered as “small,” is provided in Fleeter (1999). The National Research Council’s Aeronautics and Space Engineering Board established a Panel on Small Spacecraft Technology (1994) that defined small spacecraft as those “weighing approximately 600 kg or less.” In this work, the more commonly used definition of “small spacecraft” as those within the 0–500 kg range (Sarsfield, 1998) is adopted. This range corresponds to the categories AW and BW in the ANSI/AIAA guidelines for spacecraft design (1992). Furthermore, in these guidelines, the 500–2500 kg bin corresponds to the category CW, and masses above 2500 kg fall into the last category DW. Based on the previous discussion, the classification of spacecraft retained for the analysis is presented in Table 4.2. Spacecraft for which mass and orbit information was not available were removed from the sample. As a result, 1394 spacecraft in total remained in the sample, with 382 in the small mass bin, 546 in the medium mass bin, and 466 in the large mass bin.

4.3

Nonparametric analysis of satellite reliability by mass category

The data organized in mass bins is now treated with the Kaplan–Meier estimator as presented in Chapter 2 and the Kaplan–Meier plot of satellite spacecraft for each mass category of spacecraft is shown in Figure 4.1. The tabular data for the nonparametric results in Figure 4.1 is provided in the appendix to this chapter (Table 4.A.1). Figure 4.1 reads as follows. For example, after a successful launch, satellite reliability for the medium size category (500–2500 kg) drops to approximately 98% after 3 years on orbit. More precisely,

R(t) = 0.981

for

3.195 years ≤ t < 3.992 years.

Both small (≤ 500 kg) and large (> 2500 kg) spacecraft exhibit a reliability of 97% after 2 years. Beyond 10 years, reliability is at 88.3% for large spacecraft, 94.7%


February 18, 2011

7:31



Reliability


1.00 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0.87 0.86 0.85 0

47

Small < 500 kg Medium 500-2500 kg Large > 2500 kg 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure 4.1 Kaplan–Meier plot of spacecraft reliability for each mass category. for small ones, and 96.3% for medium ones. Because of the choice of censoring satellites at the end of their design lifetime, each line in Figure 4.1 is calculated based on satellites with these design lifetimes. For example, some small satellites had design lifetimes that were as long as 11 years. Some observations regarding the differences in failure trends between the three satellite mass categories are already apparent in Figure 4.1. For example:

r Infant mortality: small spacecraft (with a mass less than 500 kg) exhibit a significant drop in reliability during the first months following orbit insertion. For example, 6 months after orbit insertion, their reliability is already down to approximately 97.4%. This striking behavior of “infant mortality” experienced by small satellites is much less severe in the case of bigger satellites. Indeed, for the two other categories (medium and large), the drop in reliability is initially more moderate. Six months after orbit insertion, the reliability of large satellites is around 98.5%, and that of medium satellites is 98.7%.

r After roughly the first 6 months on orbit, during which infant mortality is the distinguishing feature between the three spacecraft categories, the medium and small categories exhibit a similar failure behavior associated with a miniscule decrease in reliability from year 1 to year 3. After 3 years, the reliability of small spacecraft decreases more significantly, whereas the medium spacecraft sustain a higher reliability.

r The reliability of small and medium spacecraft remains constant for a long period of time, at 94.7% from 3.7 years until 11 years for the small category, and at 96.3% from 6.1 years until 14.8 years for the medium category. This result corresponds to an absence of observed Class I on-orbit failures for these two categories of spacecraft. These portions of the nonparametric reliability



48

February 18, 2011

7:31



results are not artifacts of a decrease in the sample size because spacecraft have reached their design lifetime and were removed from the sample: 127 spacecraft remain in the small category after the last failure recorded at 3.7 years, and 137 spacecraft in the medium category after the last failure recorded at 6.1 years.

r Wear-out: large spacecraft (mass greater than 2500 kg) exhibit a distinct wearout failure behavior after 6.5 years, marked by a steep decrease in reliability. For this large category of spacecraft, this drop is more severe than during the first 6.5 years, as reflected by a change of convexity in the reliability curve approximately 6.5 years after successful orbit insertion.

r In the sample of 1394 satellites analyzed, the medium category exhibits the highest reliability of all spacecraft, always remaining above 96.3% during the observation window. The important result from Figure 4.1 is that different spacecraft mass categories do indeed have different reliability profiles and failure behaviors. In addition, Figure 4.1 indicates that the question of whether smaller or larger satellites are more (or less) reliable is ill-posed, that is, it cannot be answered without the specification of a time horizon of interest or time interval after successful orbit insertion. Once a time horizon is specified, for example, after 7 years on orbit, it can be said that small spacecraft are more reliable than large ones.

4.4

Parametric analysis of satellite reliability by mass category

Nonparametric analysis, as noted in the previous chapter, provides powerful results since the reliability calculation is not constrained to fit any particular predefined lifetime distribution. However, this flexibility makes nonparametric results neither easy nor convenient to use for different purposes, as often encountered in engineering design. In addition, some trends and failure patterns are more clearly identified and recognizable with parametric analysis. In this section, two parametric methods are presented, based on the Weibull distribution to model the reliability results of each spacecraft mass category discussed previously. The reader is referred to Chapter 3 for more details about the Weibull distribution. In this chapter, Weibull fits are first derived for the three nonparametric reliability results using the MLE procedure. These single Weibull parametric results will be shown to have an average error on the order of a single percentage point compared to the “benchmark” nonparametric results, which can be considered moderately accurate, but for the purpose of this chapter, these results are not precise enough as they miss some distinctive failure trends between the different mass categories. It was therefore decided to proceed with deriving Weibull mixture distributions for the nonparametric results and demonstrate a significant improvement in the accuracy of the parametric fits. The details are discussed next.



February 18, 2011

7:31



4.4.1

49

MLE of single Weibull fit

Details of the MLE procedure and its analytic derivation can be found in Chapter 3. When applied to the nonparametric reliability results shown in Figure 4.1, the MLE procedure yields the Weibull parameter estimates for each satellite mass category. The results are provided in Table 4.3. Consider for example the medium spacecraft category. Given Equation (3.3) and the information provided in Table 4.3, its nonparametric reliability is best approximated by the following Weibull distribution: RMedium (t) = exp −

t 18 215.6

0.4492 .

(4.1)

The values of the shape parameter (β = 0.4492) and the scale parameter (θ = 18 215.6) are the maximum likelihood estimates. With a shape parameter β < 1, the Weibull fits of spacecraft reliability provided in Table 4.3 capture the existence of infant mortality for each mass category of spacecraft. Notice that the value of the shape parameter increases as the spacecraft mass increases (i.e., 0.2519 < 0.4492 < 0.6926). This trend is in agreement with a comment made previously regarding the increased risk of infant mortality as satellite mass decreases, which was observed on the nonparametric reliability curves. Figure 4.2 shows the nonparametric reliability curve for the three mass categories, as well as the MLE Weibull fits. Note that the single Weibull models are shown only up to the last failure time observed in each mass category, that is, 3.7 years for small satellites, 6.1 years for medium satellites, and 11.8 years for large satellites.2 Figure 4.2 confirms that the Weibull distribution with the MLE parameters provided in Table 4.3 is a good fit for the nonparametric reliability of large satellites. One measure of the goodness of fit of the Weibull distribution is the average error between the nonparametric reliability results, the “benchmark” results, as termed in the previous chapter, and the Weibull fit. For example, for the large category, the average error between the nonparametric reliability curve and the Weibull fit is Table 4.3 Maximum likelihood estimates of the Weibull parameters for each mass category of spacecraft. Mass category Small (≤ 500 kg) Medium (500–2500 kg) Large (> 2500 kg)

β

θ (years)

0.2519 0.4492 0.6926

893 150.6 18 215.6 273.0

2 It is sometimes stated that the Kaplan–Meier estimator is undefined for times greater than the last observed failure time (Kalbfleisch and Prentice, 1980). Although such a statement is arguable, Figure 4.2 and the remainder of this chapter err on the side of caution by adopting this assumption.

Reliability

0.93 0 4

1 2 3 4 5 6 Time after successful orbit insertion (years)

Nonparametric estimation MLE Weibull

Medium category

7

0.84 0

0.86

0.88

0.90

0.92

0.94

0.96

0.98

1.00

1 2 3 4 5 6 7 8 9 10 11 12 13 Time after successful orbit insertion (years)


Large category

Figure 4.2 Nonparametric reliability and single Weibull fit for the three mass categories.

1 2 3 Time after successful orbit insertion (years)

0.96 0

0.97

0.98

0.99

1.00

7:31

0.94

0.95


Small category

February 18, 2011

0.96

0.97

0.98

0.99

1.00

50

Reliability


Reliability

P1: TIX/XYZ JWST062-04 Printer Name: Yet to Come




February 18, 2011

7:31



51

Table 4.4 Error between the nonparametric reliability and MLE Weibull fit for each spacecraft mass category. Mass category Small (≤ 500 kg) Medium (500–2500 kg) Large (> 2500 kg)

Average error (percentage point) 0.5287 0.2213 0.9298

0.93 percentage point. This represents an adequate accuracy for a single Weibull distribution. Table 4.4 gives the average error between the nonparametric reliability and the Weibull fit for the three mass categories. Table 4.4 shows that a single Weibull distribution provides a reasonable approximation of the nonparametric spacecraft reliability for each mass category, with an average error on the order of a single percentage point. However, for the purpose of this chapter, these results are not precise enough as they miss some distinctive failure trends between the different mass categories. It can be seen in Figure 4.2 that, for the large category for example, the parametric fit does not accurately follow the nonparametric reliability results, in particular between years 3 and 8, and between years 9 and 15 where clearly different failure trends are present and not captured by the single Weibull model. These different failure trends can be seen in the change of the convexity of the nonparametric curve around year 6.5, which reflects wear-out failures and steeper failure propensity or reliability degradation after 6.5 years on orbit. To improve the accuracy of the parametric model, mixture distributions are derived next.

4.4.2

Mixture distributions

Several distributions such as the exponential, Weibull, or lognormal distribution can be used as a basis for linear combination to generate a mixture distribution. In this section, the Weibull distribution is maintained as the basis for the parametric calculations to derive a mixture of distributions for the nonparametric spacecraft reliability of each mass category. The parametric reliability model with a mixture of 2-Weibull distributions was presented in Chapter 3, where a generalized expression for k mixture distributions is also provided. In this chapter, the calculations are restricted to k = 2 since, as will be shown shortly, the results are significantly accurate and the 2-Weibull distribution follows with notable precision the different failure trends in the nonparametric results. Increasing k does not provide a meaningful improvement in accuracy. The MLE procedure described in Section 3.5.1 provides the best models for the parameters of the 2-Weibull mixture distribution for each mass category. The results are given in Table 4.5.



52

February 18, 2011

7:31



Table 4.5 Model parameters of the 2-Weibull mixture distribution for each mass category of spacecraft. Mass category Parameter α β1 β2 θ1 θ2

Small (≤ 500 kg)

Medium (500–2500 kg)

Large (> 2500 kg)

0.9767 0.1934 12.5207 148 374 133.33 3.45

0.0128 1.0355 1.5336 0.20 103.08

0.9287 0.5190 5.5732 1569.64 10.33

For example, the resulting reliability model for large spacecraft is written according to Equation (3.21), using the appropriate parameters of Table 4.5, as follows: R(t) = 0.9287 exp −

t 1569.64

0.5190

+ 0.0713 exp −

t 10.33

5.5732 . (4.2)

Notice that for the small and large spacecraft mass categories, the mixture distribution consists of one Weibull distribution capturing infant mortality (β 1 < 1), and another one capturing an increasing failure rate (β 2 > 1), which corresponds to a wear-out failures. For the three mass categories, the new parametric fit of the reliability using a 2-Weibull mixture distribution accurately follows the nonparametric reliability, as shown in Figure 4.3. To gauge the precision accuracy between the single Weibull and the 2-Weibull mixture distributions, the average error between the benchmark nonparametric results and the parametric models is calculated. The results are given in Table 4.6. As can be seen in Table 4.6, the 2-Weibull mixture distribution is significantly more accurate than the single Weibull distribution in capturing the nonparametric spacecraft reliability results. For all mass categories, the average error for the 2-Weibull mixture distribution is reduced by over 40% compared to the average error of the single Weibull fit. In addition, for each mass category, Figure 4.4 shows the entire distribution of the residuals between the nonparametric reliability and the parametric models (single Weibull models and 2-Weibull mixtures respectively), confirming the gain in accuracy obtained by the 2-Weibull mixture distributions. The trends exhibited by the 2-Weibull mixture models are now briefly revisited in light of the failure rates of each mass category.

Reliability

0.93

0.94

0

4

0.96 0

1 2 3 4 5 6 Time after successful orbit insertion (years)

Nonparametric estimation MLE 2-Weibull mixture

Medium category

7

0.84

0.86

0.88

0.90

0.92

0.94

0.96

0.98

1.00

0

1 2 3 4 5 6 7 8 9 10 11 12 13 Time after successful orbit insertion (years)


Large category

Figure 4.3 Nonparametric reliability and 2-Weibull mixture fit for the three mass categories.

1 2 3 Time after successful orbit insertion (years)

0.97

0.98

0.99

1.00

7:31

0.95


Small category

February 18, 2011

0.96

0.97

0.98

0.99

1.00

Reliability


Reliability


DATA SPECIALIZATION: ANALYSIS BY ORBIT AND MASS 53



54

February 18, 2011

7:31



Table 4.6 Error between the nonparametric reliability and the parametric models. Average error (percentage point) Mass category

Single Weibull

2-Weibull mixture

0.5287 0.2213 0.9298

0.3029 0.1107 0.3600

Small (≤ 500 kg) Medium (500–2500 kg) Large (> 2500 kg)

4.4.3

Failure rates

The hazard function or instantaneous failure rate λ(t) is defined via the p.d.f. of the time to failure and the reliability function, as shown: λ(t) =

f (t) . R(t)

(4.3)

Furthermore, the p.d.f. of the time to failure and the reliability function are related through f (t) = −

dR(t) . dt

(4.4)

As a result, the failure rate λ(t) uniquely determines the reliability function through ⎛ R(t) = exp ⎝−

t

⎞ λ(t )dt ⎠ .

(4.5)

0

In addition, since parametric reliability functions have been derived for each satellite mass category, the resulting failure rates are computed through λ(t) = −

dR(t)/dt . R(t)

(4.6)

Figure 4.5 shows the failure rates derived from the 2-Weibull mixture models for the three spacecraft mass categories. The analysis was conducted using the parametric mixture distribution models instead of the nonparametric results for convenience and ease of visual representation of the results. The y-axis is provided in a log-scale for readability purposes. The left panel in Figure 4.5 provides a closer look at the failure rate over the short time periods through the use of a log-scale on the x-axis.

1500 1000 500 0

1500

1000

500

0 0.5

0.5

2-Weibull mixture

1 1.5 2 2.5 3 Residuals (percentage point)

Single Weibull (MLE)


3.5

3.5

4

4

0

500

1000

1500

2000

2500

3000

3500

0

0

0

0.5

0.5

2-Weibull mixture




3.5

3.5

4

4


Figure 4.4 Distribution of residuals between the nonparametric reliability and the parametric models, by mass categories.

0

0

7:31

2-Weibull mixture

2000

2000

4

2500

2500

3.5

3000

3000


3500

3500

0.5

0

0

500

1000

1500

2000

2500

3000

3500

Large category

February 18, 2011

0

500

500

Medium category



1000

1000

4

1500

1500

3.5

2000

2000


2500

2500

0.5

3000

3000

0

3500

3500

Small category


55


56

Failure rate (failures per year)

10

February 18, 2011

7:31

SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES 0

10 Small Medium Large

10

10

10


-1

-2



Small Medium Large 10

10

-3 -3

10

-2

-1

0

1

10 10 10 10 Time after successful orbit insertion (years)

0

10

-1

-2

-3

0

1 2 3 4 5 6 7 8 9 10 11 12 Time after successful orbit insertion (years)

Figure 4.5 Failure rates from the 2-Weibull mixture models, for the three satellite mass categories.

The left panel in Figure 4.5 shows that, initially, the failure rate of the small spacecraft is significantly higher than that of larger spacecraft (medium and large). This result reflects a previous observation following Figure 4.1 that small spacecraft exhibit a more pronounced infant mortality than larger ones (small spacecraft exhibit a more significant drop in reliability over the first month than larger ones, as shown in Figure 4.1). Also, in the right panel of Figure 4.5, it can be observed that the failure rate of the small spacecraft increases from 2 years to 3.5 years. Note that large spacecraft also exhibit an increase in their failure rate, from 4 to 10 years, indicative of the wear-out behavior noted previously. Finally, the failure rate of medium spacecraft remains for the most part below those of the small and large categories of spacecraft, as seen in the right panel of Figure 4.5, in accordance with the higher reliability of medium spacecraft discussed previously. In the following section, a new consideration is added to the categorization of spacecraft, in addition to their mass, namely, their orbits. The impact of orbit on spacecraft reliability is investigated, and both nonparametric and parametric reliability analyses are conducted for the new mass–orbit categories of spacecraft.

4.5

Orbit characterization

As discussed in Section 4.1, several parameters can or be correlated with the probability of failure of spacecraft. Mass was examined in the previous section. In this and the following sections, a new consideration is added to the categorization of spacecraft, in addition to their mass, namely, their orbits. Orbit type is justifiably an important contender for affecting spacecraft reliability since different design choices as well as operating environments result from the orbit type of the spacecraft. Several questions follow this observation, for example: Do spacecraft in different orbits exhibit different reliabilities and on-orbit failure behaviors? Do low Earth orbit (LEO) satellites for example exhibit different failure behaviors than spacecraft in



February 18, 2011

7:31



57

geosynchronous orbit (GEO)? Do satellites in different orbits exhibit varying degrees of infant mortality? Four orbit types are considered hereafter:

r Low Earth orbit (LEO): apogee and perigee up to 2000 km. r Medium Earth orbit (MEO): near circular orbit with apogee and perigee at about 20 000 km.

r Geosynchronous orbit (GEO): near circular orbit with apogee and perigee at about 36 000 km.

r Elliptical orbit: eccentric elliptical orbit that does not fall in the previous three categories (e.g., the Molniya orbit). Figure 4.6 shows the number of spacecraft per orbit category, out of the 1394 in the sample analyzed in this chapter, as well as the number of spacecraft per orbit category for each mass category defined in Section 4.2. The top panel of Figure 4.6 shows that the spacecraft in this study are mainly found in LEO and GEO, with 51% in LEO and 35% in GEO, versus 8% in MEO and 6% in elliptical orbits. The same observation applies to each of the mass categories, as seen in the lower panels of Figure 4.6: large satellites are mainly in LEO and GEO, and small satellites are almost exclusively confined to LEO. In addition, looking horizontally across the lower panels in Figure 4.6, it can be seen that, in LEO, about 50% of its population are small spacecraft (the other 50% are equally divided between medium and large satellites); in MEO, the population is exclusively medium spacecraft; in GEO, the spacecraft are divided between the medium and large categories (about 40% and 60% respectively) and spacecraft in elliptical orbits are primarily medium-sized. For the remainder of this chapter, the following mass–orbit categories are selected for reliability analysis: small–LEO (371 satellites), medium–LEO (162 satellites), medium–GEO (209 satellites), large–LEO (180 satellites), and large–GEO (279 satellites). The remaining categories were hardly amenable to statistical analysis.

4.6

Nonparametric analysis of spacecraft reliability by mass and orbit category

The data organized in mass–orbit bins is now treated with the Kaplan–Meier estimator as presented in Chapter 2, and the Kaplan–Meier plot of satellite reliability for each category of spacecraft is shown in Figure 4.7. The small–LEO spacecraft exhibit a similar reliability profile to the small spacecraft category in Section 4.3, which was to be expected since most of the small spacecraft are in LEO. For example, small–LEO spacecraft exhibit a significant drop in reliability during the first months on orbit, down to approximately 97.6%, and this drop is followed by another decrease in reliability between years 2 and 4 after orbit insertion, down to approximately 95%.

0

ELLIPTICAL

0

400

500

Number of satellites

300

600

209


50 100 150 200 250 300 350 400

66

109

162

Medium satellites (total: 546)

200

GEO

0

7

MEO 0

LEO

800

ELLIPTICAL

700

Figure 4.6 Satellite distribution per orbit and mass categories.


50 100 150 200 250 300 350 400

11

GEO

GEO 0

ELLIPTICAL

MEO

LEO

MEO 0

371

100

279


50 100 150 200 250 300 350 400

180

Large satellites (total: 466)

7:31

LEO

0

84

February 18, 2011

Small satellites (total: 382)

ELLIPTICAL

488


GEO

109

713

58

MEO

LEO

Orbits of the 1394 satellites selected in Section 4.2



1.00 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0.87 0.86 0.85 0.84 0.83 0.82

Reliability

1.00 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0.87 0.86 0.85 0.84 0.83 0.82 Medium LEO Medium GEO

1.00 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0.87 0.86 0.85 0.84 0.83 0.82

Large LEO Large GEO


Large satellites

7:31


Medium satellites

February 18, 2011

Figure 4.7 Nonparametric satellite reliability for each mass–orbit category.


Small–LEO satellites

Reliability


Reliability





60

February 18, 2011

7:31



Before comparing the reliability trends between LEO and GEO for the medium and large spacecraft, one must determine whether the differences between their reliability curves are statistically significant ( p < 0.05). For this purpose, two nonparametric statistical hypothesis tests are conducted: the log-rank test and the Wilcoxon test. The reader is referred to Lawless (2003) and Kvam and Vidakovic (2007) for details of these nonparametric tests. The null hypothesis considered here is that each category of satellites in LEO and GEO, the medium and large spacecraft respectively, has the same reliability function. In the case of the medium–LEO versus medium–GEO curves, the p-values provided by the log-rank and Wilcoxon tests are 0.2945 and 0.3605, respectively. These p-values clearly indicate that the difference is not statistically significant and the null hypothesis cannot be rejected. As a consequence, only the medium spacecraft category, without orbit specialization, will be considered in the remainder of this chapter. In the case of the large–LEO versus large–GEO curves, the p-values provided by the log-rank and Wilcoxon tests are 0.000 06 and 0.0002, respectively. These p-values clearly indicate that the null hypothesis should be rejected. As a consequence, it is concluded that large spacecraft have significantly different failure behaviors in LEO than in GEO. Large–LEO satellites exhibit a higher infant mortality trend than the large–GEO ones, as seen in the right panel of Figure 4.7. After 1 year on orbit, large–LEO spacecraft see their reliability drop to 92.6%, whereas large–GEO spacecraft maintain a reliability of 98.9%, a significantly large difference between these two categories of spacecraft. However, large–GEO satellites exhibit a distinct failure behavior after 6 years on orbit, marked by a steep decrease in reliability: the reliability decreases from 97.5% at 6 years to 87.8% at 12 years. Compared to previous observations in this chapter, the data specialization by mass–orbit in this section allowed both the confinement of this wear-out phenomenon to large spacecraft in GEO and the identification of infant mortality for large spacecraft in LEO.

4.7

Parametric analysis of satellite reliability by mass and orbit category

As done previously in this chapter, parametric analyses are conducted in this section for the new categories of spacecraft defined in Section 4.6. Single Weibull fits are first derived with the MLE procedure, followed by 2-Weibull mixture distributions. The MLE procedure yields the single Weibull parameters for each spacecraft mass–orbit category. The results are provided in Table 4.7. Note that the value of the shape parameter β is smaller for the LEO than the GEO spacecraft, for both the small and large categories. This result reflects an increased risk of infant mortality in LEO than in GEO, and for small spacecraft compared to large ones. As noted earlier in this chapter, single Weibull distributions are not precise enough for the purposes of this section as well, and mixture distributions are necessary to



February 18, 2011

7:31



61

Table 4.7 Maximum likelihood estimates of the single Weibull parameters for each mass–orbit category of satellites. Mass category Small–LEO Medium Large–LEO Large–GEO

β

θ (years)

0.2514 0.4492 0.7725 0.9296

1 467 278.9 18 215.6 41.9 159.5

capture the different failure trends observed in the nonparametric results. Consequently, 2-Weibull mixture distribution models are derived and their MLE parameters are given in Table 4.8. The medium-category results have been reproduced in this table to facilitate comparison to the other categories. For all the mass–orbit categories, the new parametric reliability models using a 2-Weibull mixture distribution closely follow the nonparametric reliability, as shown in Figure 4.8. The tabular data for the nonparametric results in Figure 4.8 is provided in the appendix to this chapter (Table 4.A.2). The accuracy improvement between the single Weibull and the 2-Weibull mixture distributions is provided in Table 4.9 (in terms of the average error between the benchmark nonparametric reliability results and the parametric models).

4.8

Hypotheses for causal explanations

From a statistical or actuarial perspective, the previous analyses have confirmed that spacecraft of different sizes and in different orbits exhibit different failure behaviors, but no causes have been ventured as to why these different behaviors are observed. Possible causes for these differences include considerations related to the testing Table 4.8 Maximum likelihood estimates of the parameters of the 2-Weibull mixture distribution for each mass–orbit category of spacecraft. Mass–orbit category Parameter α β1 β2 θ1 θ2

Small–LEO

Medium

Large–LEO

Large–GEO

0.9759 0.1822 12.4386 1 167 882 377.03 3.45

0.0128 1.0355 1.5336 0.20 103.08

0.9559 0.7840 1.4070 65.82 1.04

0.9057 0.4154 5.0600 46 551.01 10.21



62

February 18, 2011

7:31


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES Small–LEO satellites

Medium satellites 1.00

1.00 Nonparametric estimation MLE 2-Weibull mixture


0.99 Reliability

Reliability

0.99

0.98

0.97

0.98

0.97

0.96

0.95

0.96

0 1 2 3 4 Time after successful orbit insertion (years)

0 1 2 3 4 5 6 7 Time after successful orbit insertion (years)

Large–LEO satellites

Large–GEO satellites

1.00

1.00 Nonparametric estimation MLE 2-Weibull mixture

0.98

0.98

0.96 0.96 Reliability

Reliability

0.94 0.92 0.90 0.88

0.94 0.92 0.90

0.86 0.88

0.84 0.82


0.86 0 1 2 3 4 5 6 Time after successful orbit insertion (years)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 Time after successful orbit insertion (years)

Figure 4.8 Nonparametric reliability and 2-Weibull mixture fit for each mass–orbit category. phase of the spacecraft prior to launch, to procurement and parts selections for the spacecraft, to factors intrinsically related to the design of the system (e.g., size of the spacecraft) and factors related to the space environment, as will be discussed next. Note that the following discussion proposes hypotheses, which by their very Table 4.9 Error between the nonparametric reliability and the parametric models for each mass–orbit category of satellites. Average error (percentage point) Mass category Small–LEO Medium Large–LEO Large–GEO

Single Weibull

2-Weibull mixture

0.5854 0.2213 1.6630 1.0285

0.3120 0.1107 1.3900 0.2985



February 18, 2011

7:31



63

nature require dedicated analyses to be proved or disproved. Such analyses, although beyond the scope of this work, constitute fruitful avenues for further research within the space industry.

4.8.1

Testing

Small satellites do not benefit from the large budgets allocated to larger missions. Resource restrictions may limit the extent of testing performed on small satellites. Indeed, the procedures, facilities, and equipment, such as thermal vacuum chambers, used to test a spacecraft often remain the same regardless of the size of the spacecraft (Sarsfield, 1998). In the case of small spacecraft, extensive testing may thus have to be forfeited in order to meet budgetary constraints. Testing techniques, however, such as parts “burn-in,” when performed at appropriate stress levels and under proper environmental conditions (Gindorf et al., 1994), are essential for “remov[ing] latent defects and early failures.” The higher infant mortality exhibited by the smaller spacecraft may be due in part to differences in access to this final quality control gate that is testing, through which potential early failures are screened, detected, and fixed. In other words, less stringent and extensive testing for small satellites may contribute to their more pronounced infant mortality than large ones.

4.8.2

Procurement and parts selection

Resource constraints, which are likely more acute for small satellites, have resulted in an increased adoption of commercial off-the-shelf (COTS) parts in the design of satellites. While COTS parts must undergo a series of tests to become suitable for operation in the space environment (“space rated”), their frequent use may still represent a challenge in terms of reliability. As they remain generally less radiationhardened than electronic parts obtained from traditional suppliers, COTS parts are particularly sensitive to the effects of high-energy particles in the space environment. This vulnerability can manifest itself through single-event effects (SEEs), affecting memories, power devices, or control logic devices, which may result in spacecraft failure. In addition, COTS parts are often rated to operate in a narrow range of temperatures which does not always match the thermal requirements associated with the space environment (Jiang et al., 2009; Fortescue et al., 2003). Despite intensive testing procedures, COTS parts may remain more likely to fail due to the extreme thermal conditions of the space environment for which they were not initially designed. In addition to temperature, humidity conditions constitute another environmental factor that can alter the normal behavior of electronic components. For example, plastic-encapsulated microcircuits (PEMs) have recently gained popularity in space applications due to their cost and availability benefits. However, despite their increasing reliability, they remain more susceptible to moisture absorption than the traditional hermetically sealed microcircuits (HSMs) (Jiang et al., 2009).



64

February 18, 2011

7:31



As a result, they may pose potential reliability concerns if additional testing is not properly conducted. In short, a difference in the extent of reliance on COTS parts may contribute to the difference in failure behavior between small and large spacecraft, especially during their first exposure to the space environment (i.e., in terms of infant mortality).

4.8.3

Factors intrinsically related to the design of the spacecraft

The mass and geometric constraints (volume and other dimensions) imposed on small spacecraft can translate into several potential contributions to failures. For example:

r Small spacecraft, for which weight represents a critical parameter, cannot afford to have as much redundancy as larger spacecraft. They are thus often based on “single-string” designs, which can turn a simple anomaly into the complete loss of the spacecraft.

r The design of small spacecraft relies on a greater packaging density, which can expose certain parts to higher temperatures (from the electronics and power subsystem), resulting in an increased risk of failure (Sarsfield, 1998).

r “By its nature, a small spacecraft offers less natural shielding” than large spacecraft (Sarsfield, 1998), and is therefore more exposed to the effects of cumulative radiation. These effects include: (1) the total ionizing dose (TID), which can affect capacitors, transistors, etc., and result in variations in leakage current and threshold voltage; and (2) nonionizing radiation such as displacement damage dose (DDD) that “degrades the performance of solar cells” and other parts of the spacecraft. It is more difficult to harden against DDD, therefore the use of shielding [. . .] is used to mitigate its effects” (Barth, 2005). As a result, small spacecraft that do not benefit from sufficient shielding may be particularly vulnerable to radiationinduced failures. Factors that can potentially affect the failure behavior of large spacecraft include the following:

r Scaling up spacecraft subsystems to fulfill the requirements of large spacecraft may pose distinct technological challenges, and thus distinct failure behaviors, that are not encountered in their smaller counterparts. For example, larger and heavier spacecraft experience higher structural and electrical loads than smaller satellites. Power systems generate heat that may accelerate the physical degradation of parts over time. These phenomena may contribute to the increased wear-out failures experienced by large–GEO spacecraft and the corresponding significant decrease in reliability after 6 years on orbit.



February 18, 2011

7:31



65

r The complexity of large spacecraft may also contribute to some infant mortality failures in the following manner. Large spacecraft with multiple instruments and subsystems require intensive wiring and an increased number of interfaces, adding more potential failure points than their smaller counterparts. As the number of connections between subsystems increases, the integration process becomes more delicate. This in turn may increase the likelihood of human errors, which can translate into failures observed early during the on-orbit life of large spacecraft.

4.8.4

Factors related to the space environment

The space environment is likely an important causal factor for the differences in failure behaviors between spacecraft in different orbits. The way in which the space environment may affect spacecraft failures and consequently their reliability is discussed next:

r Upper atmosphere: a significant difference in the space environment between LEO and GEO is the presence of a rarefied atmosphere in the LEO environment. The spacecraft orbit and service life can be impacted by the upper atmosphere, with phenomena such as atmospheric drag and heat, and the “chemically corrosive effects of highly reactive elements such as atomic oxygen” (Walterscheid, 1999), which is the major element in the atmosphere between roughly 200 and 600 km. Between 120 and 600 km, atmospheric drag is a significant parameter affecting satellite orbit and must be counteracted for a spacecraft to remain in an operational orbit for a long period of time. Atmospheric drag may not be a direct cause of spacecraft failures due to its progressive nature, but its cumulative effect can hasten the decommissioning dates for LEO spacecraft. Atmospheric drag effects can also worsen the impact of anomalies or failures that would be minor under other conditions. For example, failures in the attitude control, propulsion, or communication subsystems can lead to the loss of the spacecraft, as the reaction window for the recovery team on the ground is shortened by drag losses and draginduced torques. These effects related to the upper atmosphere might bring about failure mechanisms peculiar to the LEO environment, and may contribute to the differences in failure behavior and reliability degradation observed between large–LEO and large–GEO spacecraft.

r Plasma and arcing: in both LEO and GEO, spacecraft pass though plasma and can become charged to high negative voltages as a consequence (Koons, 1999). As a consequence, electrostatic discharge (ESD) or arcs can occur and damage the surface components, or disrupt electronics components through arcing-induced electromagnetic interference (EMI) (Koons, 1999, citing Robinson, 1989). Different types of plasma can be found in LEO or GEO



66

February 18, 2011

7:31



environments: plasma in LEO is dense and low energetic, when it is scarce, but highly energetic in GEO (Bedingfield et al., 1996). Such differences lead to different arcing mechanisms, and consequently they may cause different failure behaviors and reliability degradation. The arcing phenomenon is considered a major concern for GEO spacecraft, particularly when spacecraft “emerge from an eclipse period into a solar storm” (Rodiek et al., 2008). The reader interested in more details about spacecraft arcing is referred to Katz et al. (1998), Ferguson et al. (1999), and Frezet et al. (1989).

r Radiation: radiation of different types and energies is encountered in different orbits and may contribute causal factors for explaining differences in reliability and on-orbit failures between spacecraft in different orbits. How these factors impact the observed reliability profiles of different categories of spacecraft cannot yet be ascertained, but it deserves to be acknowledged. For example, the Van Allen radiation belts, which are of concern to LEO and occasionally MEO spacecraft, consist primarily of trapped electrons and protons within the Earth’s magnetosphere (Schulz and Vampola, 1999). Their differential effect on spacecraft failures is ambiguous: on the one hand, their cumulative dose effect can result in specific failure modes in spacecraft; on the other hand, their presence can shield to some extent spacecraft orbiting below them from highly energetic solar particle events or galactic cosmic rays, thus preventing other specific failure modes. “Solar particles events (SPE) occur in association with solar flares [and they consist of] rapid increases in the flux of energetic particles (∼1 MeV to above GeV) lasting from several hours to several days” (Gorney, 1999). SPE can lead to the degradation of solar arrays or electro-optical sensors. These space environment considerations, radiation belts, SPE, galactic cosmic rays (GCRs), and high-energy charged particles (∼0.1 to 10 GeV, and occasionally significantly more) can cause single-event upsets (SEUs), latchup (SEL), or burn-out (SEB) in spacecraft electronics (Blake, 1999), which can result in erratic spacecraft commands/ behavior and loss.

r Thermal and power cycling: in addition to the previous space environment effects, another important difference between LEO and GEO spacecraft is the frequency and extent to which they experience eclipses, and thus the degree to which they experience thermal and power cycling. For example, a typical LEO spacecraft experiences one eclipse period per orbit, about 15 eclipses per day, and about 36 minutes of shadowing at most (McDermott, 1999). By contrast, a GEO spacecraft experiences “two 45-day eclipse periods per year with eclipses lasting no more than 72 minutes each day” (McDermott, 1999). Such cycles affect the thermal and power subsystems: daylight and eclipse periods modify the temperature of the spacecraft and require switching between the use of the solar arrays and the batteries. Thermal expansion and contraction effects can cause component fatigue and in-and-out eclipse cycles affect the use and depth of discharge of batteries. Thermal and power



February 18, 2011

7:31



67

cycling may contribute to the difference in failure behavior between LEO and GEO spacecraft, since the latter are subjected to a significantly higher number of power and thermal cycles during their lifetime than their counterparts in GEO.

4.A 4.A.1

Appendix: Tabular data and confidence interval analysis Tabular data for the nonparametric reliability results in Figure 4.1 and Figure 4.8

Table 4.A.1 Tabular data for the nonparametric results in Figure 4.1. Small category Failure time ti (years) 0.0027 0.0082 0.0110 0.0137 0.0246 0.9144 2.8830 2.9377 3.4387 3.6879

ˆ i) R(t 0.9921 0.9869 0.9816 0.9763 0.9737 0.9705 0.9663 0.9620 0.9546 0.9471

Medium category Failure time ti (years)

ˆ i) R(t

0.0055 0.0329 0.1451 0.1862 0.2163 0.3368 0.4572 1.9767 3.1951 3.9918 5.9713 6.1246

0.9982 0.9963 0.9944 0.9925 0.9906 0.9887 0.9867 0.9845 0.9809 0.9772 0.9703 0.9633

Large category Failure time ti (years)

ˆ i) R(t

0.0027 0.0192 0.1752 0.2930 0.3504 0.3587 0.8460 0.8597 0.8679 0.9966 1.9192 1.9521 1.9822 3.0719 4.8679 4.9199 6.6502 6.7680 7.0637 7.9863 8.0684 8.9473 9.4593 10.0862 11.8385

0.9979 0.9957 0.9931 0.9905 0.9877 0.9850 0.9821 0.9792 0.9764 0.9734 0.9701 0.9667 0.9633 0.9593 0.9548 0.9503 0.9445 0.9386 0.9323 0.9252 0.9178 0.9092 0.8995 0.8831 0.8572



68

February 18, 2011

7:31



Table 4.A.2 Tabular data for the nonparametric results in Figure 4.8. Small–LEO Failure time ti (years)

ˆ i) R(t

0.0027 0.0082 0.0110 0.0137 0.0246 2.8830 2.9377 3.4387 3.6879

0.9946 0.9892 0.9837 0.9783 0.9756 0.9713 0.9670 0.9595 0.9520

4.A.2

Medium Failure time ti (years)

ˆ i) R(t

0.0055 0.0329 0.1451 0.1862 0.2163 0.3368 0.4572 1.9767 3.1951 3.9918 5.9713 6.1246

0.9982 0.9963 0.9944 0.9925 0.9906 0.9887 0.9867 0.9845 0.9809 0.9772 0.9703 0.9633

Large–LEO Failure time ti (years)

ˆ i) R(t

0.0192 0.1752 0.2930 0.3587 0.8460 0.8597 0.8679 1.9521 1.9822 4.8679

0.9944 0.9849 0.9745 0.9631 0.9508 0.9384 0.9258 0.9026 0.8789 0.8239

Large–GEO Failure time ti (years)

ˆ i) R(t

0.0027 0.3504 0.9966 1.9192 3.0719 4.9199 6.6502 6.7680 7.0637 7.9863 8.0684 8.9473 9.4593 10.0862 11.8385

0.9964 0.9927 0.9888 0.9847 0.9802 0.9751 0.9688 0.9624 0.9557 0.9481 0.9403 0.9313 0.9212 0.9044 0.8778


The Kaplan–Meier estimator provides a maximum likelihood estimate of reliability, but does not inform about the dispersion about the reliability estimate. This dispersion is captured in the 95% confidence interval, and the reader is referred to Chapter 2 for details about confidence intervals. The confidence intervals for the mass categories are shown in Figure 4.A.1, while the confidence intervals for the mass–orbit categories are shown in Figure 4.A.2. Figure 4.A.1 shows for example that the reliability of medium satellites 5 years after orbit insertion will fall between 96.3% and 99.2% with a 95% likelihood (confidence interval). In addition, the most likely reliability estimate is at t = 5 years ˆ = 5years) = 94.7%. Note that for the small satellites is the Kaplan–Meier result R(t ˆ the dispersion of R(ti ) around R(ti ) increases with time. This increase in dispersion can be seen in Figure 4.A.1 and Figure 4.A.2 by the growing gap between the Kaplan–Meier estimated reliability and the confidence interval curves. This phenomenon illustrates the increasing uncertainty or loss of accuracy of the statistical analysis of satellite reliability with time resulting from the decreasing sample size (see the discussion in Section 4.1 regarding the limitation of data specialization for satellite reliability analysis).

1.00 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0.87 0.86 0.85

Reliability


Nonparametric estimation 95% confidence interval

0.74

0.82 0.80 0.78 0.76

0.90 0.88 0.86 0.84

1.00 0.98 0.96 0.94 0.92

0



Large category

Figure 4.A.1 Nonparametric satellite reliability with 95% confidence intervals for each mass category.

0.88

0.89

0.90

Medium category

7:31

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00

February 18, 2011



Small category

Reliability


Reliability





70

February 18, 2011

7:31


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES Medium satellites 1.00 0.99 0.98 0.97 0.96 Reliability

Reliability

Small–LEO satellites 1.00 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0.87 Nonparametric estimation 0.86 95% confidence interval 0.85 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

0.95 0.94 0.93 0.92 0.91 0.90

Nonparametric estimation 95% confidence interval 0.88 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years) 0.89

Large–LEO satellites

Large–GEO satellites

1.00

1.00

0.95

0.98 0.96 0.94

0.85

Reliability

Reliability

0.90

0.80 0.75

0.90 0.88 0.86

0.70 0.65

0.92

0.84 Nonparametric estimation 95% confidence interval

0.60 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Nonparametric estimation 95% confidence interval 0.80 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years) 0.82

Figure 4.A.2 Nonparametric satellite reliability with 95% confidence intervals for each mass–orbit category.



February 17, 2011

7:56


5

Spacecraft subsystem reliability1 In this chapter, the previous statistical analysis of spacecraft reliability is extended to include spacecraft subsystems, that is, the analysis is narrowed down from the system-level to the subsystem-level failures, and reliability results, nonparametric and parametric, are derived for spacecraft subsystems. The two broad questions addressed in this chapter are: (1) What are the reliability profiles of various spacecraft subsystems? (2) To what extent does each subsystem contribute to the overall failures of spacecraft? The answer to the second question constitutes a comparative analysis of subsystems failure, from an actuarial perspective, and allows for example the identification of culprit subsystems driving spacecraft unreliability. The results provided here should prove helpful to spacecraft manufacturers by allowing them to hone in on problematic subsystems that would benefit most from increased testing and reliability improvements. The remainder of this chapter is organized as follows. The subsystems identified in the analysis are presented in Section 5.1. Nonparametric analysis of satellite subsystems failure data is conducted in Section 5.2, and Weibull parametric fits are presented in Section 5.3. Finally, the comparative analysis or extent to which each subsystem contributes to the overall failures of spacecraft is provided in Section 5.4.

5.1

Spacecraft subsystem identification

The statistical failure data analysis at the subsystem level is enabled by, and confined to, the subsystems identified in the database (see the discussion in Section 2.2 for 1 This chapter is based on an article written by the authors and published in Reliability Engineering and System Safety (Castet and Saleh, 2009b).




72

February 17, 2011

7:56



information about the database). The following spacecraft subsystems are recognized in the database: 1. Gyro/sensor/reaction wheel (hereafter referred to as Gyro). 2. Thruster/fuel (Thruster). 3. Beam/antenna operation/deployment (Beam). 4. Control processor (CP). 5. Mechanisms/structures/thermal (Mechanisms). 6. Payload instrument/amplifier/on-board data/computer/transponder (Payload). 7. Battery/cell (Battery). 8. Electrical distribution (ED). 9. Solar array deployment (SAD). 10. Solar array operating (SAO). 11. Telemetry, tracking, and command (TTC). For example, a traveling-wave tube (TWT) is categorized under subsystem 6 in the previous list, and the solar array drive, if its failure leads to the nondeployment of the solar arrays, would be categorized under subsystem 9 is the list. These categories are not necessarily spacecraft “subsystems” in the traditional meaning of the word. For example, the “Battery/cell” and “Electrical distribution” are part of the spacecraft “electrical power subsystem” (EPS) but they are clearly identified in the database when they cause a spacecraft failure. As a result, there is in some of these categories a finer resolution for causes of spacecraft failure than the subsystem level (i.e., in some cases, it is the failure at the sub-subsystem level that is identified). Notice also that the “Solar array deployment” is a one-shot “subsystem” or, more precisely, a one-shot phase of the solar array sub-subsystem. For convenience, all these 11 items in the list are referred to as “subsystems.” Descriptions of these subsystems can be found in any textbook on spacecraft systems engineering such as that by Fortescue et al. (2003) or Wertz and Larson (1999). When the culprit subsystem that led to the failure of the satellite cannot be identified, the failure of the spacecraft is ascribed to an “Unknown” category in the database. Only the Beam/antenna operation/deployment subsystem exhibits no Class I failure in the data set (see Section 2.2 for a discussion of classes of failures). Thus the following study is confined to the 10 remaining subsystems plus the Unknown category.

5.2

Nonparametric reliability analysis of spacecraft subsystems

The initial sample of 1584 spacecraft is analyzed using the statistical tools presented in Chapter 2. The subsystem failure data is treated with the Kaplan–Meier


February 17, 2011

7:56


SPACECRAFT SUBSYSTEM RELIABILITY

73

estimator (2.10), and the Kaplan–Meier plots of the reliability of all the spacecraft subsystems listed in the previous section are derived. In addition, when (2.12) and (2.13) are applied to the data from the sample, the 95% confidence intervals are obtained. The nonparametric reliability results of the spacecraft subsystems considered here along with their 95% confidence intervals are shown in Figure 5.1 and Figure 5.2.

Thruster / Fuel

Gyro / Sensor / Reaction wheel 1.000

1.000

0.995

0.998 0.996

0.990 0.985

Reliability

Reliability

0.994

0.980 0.975

0.992 0.990 0.988 0.986

0.970

0.984 0.965


0.960 0

0.982 0.980 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)


Control Processor

Mechanisms / Structures / Thermal

1.000

1.000 0.999

0.998


Reliability

0.996 0.994 0.992

0.996 0.995 0.994 0.993

0.990

0.992 0.988

0.991



Payload instrument / Amplifier / On-board data / Computer / Transponder

Battery / Cell 1.000

1.000 0.995

0.999

0.990

0.997 0.996 0.995 0.994

Reliability

0.998 Reliability


0.985 0.980 0.975

0.993

0.970



Figure 5.1 Spacecraft subsystem reliability with 95% confidence intervals (1/2).



74

February 17, 2011

7:56


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES Solar array deployment

Electrical distribution 1.000

1.0000

0.998

0.9995

0.996

0.9990 Reliability

Reliability

0.994 0.992 0.990 0.988

0.982 0


0.9975



Solar array operating 1.000 0.998 0.996 0.994 0.992

Reliability

Reliability

0.9980

0.9970

0.986 0.984

0.9985

0.990 0.988 0.986 0.984 0.982 0.980 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Telemetry, tracking and command 1.000 0.998 0.996 0.994 0.992 0.990 0.988 0.986 0.984 0.982 0.980 0.978 0.976 0.974 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure 5.2 Spacecraft subsystem reliability with 95% confidence intervals (2/2). Figure 5.1 and Figure 5.2 read as follows. Consider the “Gyro” subsystem, its reliability being shown in the upper left corner of Figure 5.1. After a successful launch, the reliability of this subsystem drops to approximately 99.5% after 4 years on orbit. More precisely, ˆ = 0.9948 R(t)

for

1146 days ≤ t < 1967 days

that is, 3.137 years ≤ t < 5.385 years. In addition, the reliability of this subsystem will fall between 99.1% and 99.9%, its 95% confidence interval, over this period of time. ˆ and confidence This same “reading grid” regarding the estimated reliability R(t) interval applies to all the other subsystems of Figure 5.1 and Figure 5.2. For example, the reliability of the TTC subsystem (lower right corner of Figure 5.2) drops to 0.984 after 8 years on orbit, with a 95% confidence interval between 97.6% and 99.1%. Notice the particular nonparametric reliability of the SAD, a constant, which is due to the one-shot nature of this “subsystem” (or, more precisely, to this phase of the solar array subsystem). A general observation can be made regarding these


February 17, 2011

7:56



75

nonparametric reliability results, namely, that spacecraft subsystems are notably reliable, all maintaining reliability above than 98% after 15 years on orbit (and above 97% for the lower bound of the 95% confidence interval). However, the collective failure contributions of these subsystems lead to spacecraft reliability in the 80% to 90% range as seen in the previous chapters, a deplorable and costly situation, considering that these high-value assets often cost hundreds of millions of dollars to design and launch, and they do not benefit from physical access and maintenance to remedy on-orbit failures. Consequently, improvements to spacecraft subsystem reliability are warranted, and the nonparametric results in Figure 5.1 and Figure 5.2 provide a first indication of possible subsystem failure patterns to target and remedy. Notice for example the distinct and marked infant mortality failures of the Thruster and TTC subsystems, which could be eliminated through improved testing or burn-in procedures

5.3

Weibull modeling of spacecraft subsystem reliability

Weibull parametric models for spacecraft subsystem reliability are provided in this section. Weibull distributions have been shown previously to be adequate for modeling spacecraft reliability, and they are more appropriate than the lognormal and the exponential distributions. The same observation can be extended to spacecraft subsystems. For example, Figure 5.3 shows the Weibull plot for the Thruster subsystem reliability.

-4.0 -4.5

Empirical data Linear least squares fit

-5.0 ln[-ln R(t)]


-5.5 -6.0 -6.5 -7.0 -7.5 -8.0 -6

-5

-4

-3

-2 ln(t)

-1

0

1

2

Figure 5.3 Weibull plot of Thruster subsystem reliability (shown above are the data points and linear least squares fit).



76

February 17, 2011

7:56



Table 5.1 Maximum likelihood estimates of the Weibull parameters for subsystem reliability. Subsystem Gyro Thruster CP Mechanisms Payload Battery ED SAD SAO TTC

β

θ (years)

0.7182 0.3375 1.4560 0.3560 0.8874 0.7460 0.5021 — 0.4035 0.3939

3831 6 206 945 408 2 308 746 7983 7733 169 272 — 1 965 868 400 982

The data points in Figure 5.3 are well aligned,2 which provides a good indication that the Weibull distribution provides a good fit for the Thruster subsystem reliability (see the discussion in Section 3.2 about probability plots and their use). Similar results are obtained for the other subsystems. Reliability of spacecraft subsystems, with the exception of the SAD, can be adequately modeled by Weibull distributions. The goodness of fit of this distribution is addressed later in this section. The MLE procedure, introduced previously in Section 3.3, is applied to the failure and censored times of each subsystem to determine the maximum likelihood Weibull parameters of each subsystem’s reliability model. The results for the Weibull shape and scale parameters are given in Table 5.1. The information in Table 5.1 reads as follows. For example, the Weibull reliability model for the Gyro subsystem is written as follows: 0.7182 t . RGyro (t) = exp − 3831

(5.1)

The values of the shape parameter (β = 0.7182) and the scale parameter (θ = 3831) are the maximum likelihood estimates. Note that no values of the Weibull parameters are provided for the SAD subsystem. As discussed previously and seen in Figure 5.2, the “SAD” is a one-shot “subsystem” and a Weibull fit is not meaningful in this case. A Weibull fit can also be conducted on the data assigned to the “Unknown” category mentioned in Section 5.1. The resulting Weibull parameters are β = 0.4011 and θ = 5 836 474 years. The important result in Table 5.1 is that all spacecraft subsystems, with the exception of the CP, suffer from infant mortality (shape parameter β < 1). This 2 The

linear least squares fit has a coefficient of determination R2 = 0.9670 for the Thruster subsystem.


February 17, 2011

7:56



Reliability


1.000 0.998 0.996 0.994 0.992 0.990 0.988 0.986 0.984 0.982 0.980 0.978 0.976 0.974 0

77

Nonparametric estimation MLE - Weibull 95% confidence interval


Figure 5.4 Nonparametric reliability and Weibull fit for the TTC subsystem. finding has important implications for the space industry and should prompt serious consideration for improved subsystem testing and burn-in procedures. Figure 5.4 shows the nonparametric reliability curve (with the 95% confidence interval) for the TTC subsystem, as well as the MLE Weibull model. Figure 5.4 provides a visual confirmation that the Weibull distribution with the MLE parameters given in Table 5.1 is a good fit for the TTC nonparametric (benchmark) reliability. Similar results are obtained for the other subsystems, and they are reflected in the maximum and average errors over 15 years between the nonparametric reliability curve and the Weibull model shown in Table 5.2. For example, for Table 5.2 Error between the nonparametric and Weibull reliability for each subsystem. Maximum error Subsystem Gyro Thruster CP Mechanisms Payload Battery ED SAD SAO TTC

Average error

(percentage point) 0.37 0.18 0.22 0.21 0.09 0.62 0.19 –– 0.31 0.23

0.14 0.08 0.06 0.07 0.03 0.15 0.07 –– 0.13 0.10



78

February 17, 2011

7:56



the TTC subsystem, the maximum error between the nonparametric reliability curve and the Weibull fit is 0.23 percentage point, and the average error is 0.10 percentage point. This represents a remarkable accuracy for only a two-parameter (Weibull) distribution. Improved accuracy is likely superfluous, but if needed it can be obtained as done previously through the use of mixture distributions.

5.4

Comparative analysis of subsystem failures

A comparative analysis of subsystem failure is provided in this section and the culprit subsystems driving spacecraft unreliability are identified. More specifically, the relative contribution of each subsystem to the failure of the spacecraft in the sample is quantified. In addition, a time dimension is added to this analysis by investigating the evolution over time of the relative contribution of each subsystem to the loss of spacecraft. The percentage contribution of subsystem j to the failure of a spacecraft is then defined as rj =

number of spacecraft failures due to the failure of subsystem j . number of spacecraft failures

(5.2)

For each subsystem j identified in the database, its probability of leading to the failure of the satellite, Pˆ subsystem,j , is calculated based on the estimated reliability of the subsystem obtained by the Kaplan–Meier estimator in Section 5.2: Pˆsubsystem,j = 1 − Rˆ subsystem,j .

(5.3)

Then the probability of failure of a satellite is computed as follows: Pˆspacecraft = 1 − Rˆ spacecraft

(5.4)

where Rˆ spacecraft is the nonparametric spacecraft reliability obtained with the Kaplan–Meier estimator (shown in Figure 2.3). In the case of the present study where the n = 11 subsystems are considered in series (each subsystem needs to be operational for the spacecraft to be operational), the percentage contribution of subsystem j to the failure of a spacecraft can be rewritten as n−1 (−1)i si ˆ Pˆspacecraft (5.5) r j = Psubsystem,j 1 + i + 1 i=1 ˆ where si is the sum of all the possible combinations of products of i Psubsystem,k , k = j n−1 ( terms).3 i 3 If

the subsystem failure probabilities Pˆsubsystem,j are small enough, r j ≈ Pˆsubsystem,j / Pˆspacecraft .


February 17, 2011

7:56



79

30

Gyro Thruster TTC

25

20

15

10

5

0


35

30

Mechanisms CP SAO

25

20

15

10

5

0


Percent contribution to spacecraft failure (cumulative, %)

35



Deriving rj for all subsystems addresses the second question of this chapter, that is, to what extent each subsystem contributes to the overall failure of the spacecraft. The results of the analysis can be displayed in one figure, showing all the rj for j = 1 to 11 as a function of time. Doing so, however, would result in an exceedingly cluttered figure. For readability purposes, the results are split into four panels in Figure 5.5 and Figure 5.6. The full names of the subsystems and their abbreviations are provided in Section 5.1. Figure 5.5 shows the evolution over time of the contribution of each subsystem to the loss of the spacecraft. For example, it can be seen in the lower left quadrant of Figure 5.5, that the CP contributes approximately 6% to the total failures of spacecraft over 15 years. Similarly in the upper left quadrant of Figure 5.5, it is



35

30

Payload Battery ED

25

20

15

10

5

0


35 SAD Unknown 30

25

20

15

10

5

0


Figure 5.5 Relative contributions of various subsystems to satellite failure.



80

February 17, 2011

7:56



observed that the Gyro and TTC are the major contributors to spacecraft failures with respectively 20% and 15% of satellite failures due to these subsystems over a period of 15 years. It is interesting to note the switch in “failure leadership” between the Gyro and TTC subsystems around year 10, as seen in the upper left quadrant of Figure 5.5. The TTC is the lead contributor to satellite failure over the first 10 years on orbit, with a relative contribution hovering around 20%. The failures due to the Gyro remain around 12% between year 1 and year 6 on orbit, and then they clearly ramp up, starting around year 6, and overtake the relative contributions of the TTC subsystem to spacecraft failure. This result is in line with the nonparametric reliability results of the Gyro and TTC seen in Figure 5.1 and Figure 5.2, where TTC failures are concentrated in the early years of the spacecraft on orbit and subsiding after 8 years, whereas the Gyro has a more pronounced tendency to fail after 6 years on orbit. Another interesting trend in the upper left quadrant of Figure 5.5 is the contribution of the Thruster subsystem to spacecraft failures, which is only second to leadership in failure of the TTC subsystem for the first 6 years on orbit, contributing 18% to 22% of all spacecraft failures, before dropping steadily to roughly 10% over a 15-year period. This result also reflects the marked infant mortality experienced by this subsystem and seen in its nonparametric reliability result in Figure 5.1. These results clearly mark the TTC, Gyro, and Thruster subsystems as the major culprits driving spacecraft unreliability and the ones that would benefit most from reliability improvements. The lower right quadrant of Figure 5.5 shows for example that unknown causes account for 4% to 8% of spacecraft failures (which is indicative of the extent of spacecraft state of health (SOH) monitoring and telemetry points). Also of interest is the evolution of the battery contribution to spacecraft failure shown in the upper right quadrant of Figure 5.5. Two clear increases are observed in rbattery : the first one around year 3 on orbit when satellite failures due to batteries ramp up from 2% to 10% by year 5; and the second one around year 14 when satellite failures due to batteries ramp up from 5% to 14% by year 15 (this information can also be seen in Figure 5.1). These observations are likely indicative of two different failure modes in batteries, and, as such, they can provide some guidance to electrical engineers working on spacecraft power storage and the corresponding reliability testing program. Figure 5.6 provides a more readable version of Figure 5.5. Instead of the evolution over time of rj , Figure 5.6 provides a snapshot or static picture of the subsystems’ contributions to spacecraft failures at four discrete points in time: after 30 days, after 1 year, after 5 years, and after 10 years on orbit. Figure 5.6 in effect represents vertical cuts across Figure 5.5, and while the dynamical information portrayed in these figures is lost, readability and accuracy (or finer resolution) are gained at the discrete points in time selected. The observations made on Figure 5.5 can be found in Figure 5.6. In addition, notice in the upper left quadrant of Figure 5.6 that the solar array (deployment and operating) and TTC account respectively for 20% and 28% of the failures of the first 30 days on orbit. Thus spacecraft infant mortality, as discussed in Chapter 2



February 17, 2011

7:56


SPACECRAFT SUBSYSTEM RELIABILITY t = 30 days Unknown 10%

SAO 10%

t = 1 year

Gyro 5%

Unknown 6%

Thruster/Fuel 16% CP 0%

TTC 28%

SAD 10%

Mechanisms 5% Payload 0% Battery/Cell 5% Electrical distribution 11%

Gyro 11%

TTC 25%

Thruster/Fuel 23% CP 0%

SAO 11% SAD 4%

Electrical distribution 7%

t = 5 years Unknown 6%

81

Battery/Cell 2%

Mechanisms 9% Payload 2%

t = 10 years Unknown 5%

Gyro 10%

TTC 21%

Gyro 18%

TTC 21%

Thruster/Fuel 18%

Thruster/Fuel 13% SAO 7% SAD 2% Electrical distribution 9%

CP 4% Mechanisms 10%

Payload Battery/Cell 3% 10%

SAO 9% SAD 2%


Battery/Cell 6%

CP 6% Mechanisms Payload 6% 4%

Figure 5.6 Subsystem contributions to spacecraft failures after 30 days, 1 year, 5 years, and 10 years on orbit. and Chapter 3, is driven to a large extent by these two subsystems, followed by the Thruster subsystem, during the first month on orbit. In subsequent chapters, a more detailed approach to the degradation behavior of spacecraft subsystems is developed by accounting for and analyzing their anomalies and partial failures, that is, failures of different severity, not just Class I (total) failures.



February 17, 2011

10:8


6

Time to anomaly and failure of spacecraft subsystems: exploratory data analysis 6.1

Introduction

The present chapter is a turning point in this work. The previous chapters dealt with the reliability of spacecraft and spacecraft subsystems, a critical design attribute for high-value assets. The events considered were catastrophic failures (Class I) that lead to the complete loss of the spacecraft. As a result, only two states were considered, operational and failed, and the (sub)systems were analyzed and modeled as being in one of these two states. In reality, many engineering artifacts, spacecraft included, can experience failure events of varying severity, and thus transition from fully operational to various states of partial degradation, not necessarily complete failures. This chapter extends the previous analysis of reliability, in its traditional binary-state understanding, to account for spacecraft anomalies and failures of various severity. As will be shown in this chapter, partial failures of different severity constitute a significant portion of anomalous events that spacecraft experience on orbit, and as such their analysis provides additional and important pieces of information toward an understanding of spacecraft and subsystem failure behavior and propensity. This chapter can be characterized as an exploratory data analysis of anomaly and failure distributions of the sample analyzed in this work (see Chapter 2 for details), as well as the time to anomaly – of varying severity – of each spacecraft subsystem. Unlike previous chapters, this chapter brings no formal statistical tools to bear on the analysis. Formal multi-state failure analysis of spacecraft subsystems is provided in Chapter 7. This chapter serves as an easy transition between the formal Spacecraft Reliability and Multi-State Failures: A Statistical Approach, First Edition. Joseph Homer Saleh and Jean-François Castet. © 2011 John Wiley & Sons, Ltd. Published 2011 by John Wiley & Sons, Ltd. ISBN: 978-0-470-68791-8



84

February 17, 2011

10:8



binary-state understanding of reliability conducted in the previous chapters and the formal multi-state failure analysis in the following chapters.

6.2

Anomaly and failure events

The same database and spacecraft sample used in the previous chapters are also used in this chapter. Recall that the sample consists of 1584 Earth-orbiting spacecraft successfully launched between January 1990 and October 2008. The spacecraft subsystems identified in the database are the following (details can be found in Chapter 5): 1. Gyro/sensor/reaction wheel (hereafter referred to as Gyro). 2. Thruster/fuel (Thruster). 3. Beam/antenna operation/deployment (Beam). 4. Control processor (CP). 5. Mechanisms/structures/thermal (Mechanisms). 6. Payload instrument/amplifier/on-board data/computer/transponder (Payload). 7. Battery/cell (Battery). 8. Electrical distribution (ED). 9. Solar array deployment (SAD). 10. Solar array operating (SAO). 11. Telemetry, tracking, and command (TTC). Details about the subsystems and the “Unknown” category can be found in Chapter 5. It should be pointed out, as explained in Chapter 5, that these categories are not necessarily spacecraft “subsystems” in the traditional meaning of the word. For example, the “Battery/cell” and “Electrical distribution” subsystems are part of the spacecraft “electrical power subsystem (EPS)” but they are clearly identified in the database when they cause a spacecraft failure. As a result, there is in some of these categories a finer resolution for causes of spacecraft failure than the subsystem level (i.e., in some cases, it is the failure at the sub-subsystem level that is identified). Notice also that the “solar array deployment” is a one-shot “subsystem” or, more precisely, a one-shot phase of the solar array sub-subsystem. The statistical analysis is enabled by, and confined to, the classes of (partial) failures identified in the database. These classes of failures, as defined in the database, are listed next by increasing order of the failure severity:

r Class IV: minor/temporary/repairable failure that does not have a significant permanent impact on the operation of the satellite or its subsystems.



February 17, 2011

10:8


TIME TO ANOMALY AND FAILURE OF SPACECRAFT SUBSYSTEMS

85

r Class III: major nonrepairable failure that causes the loss of redundancy to the operation of a satellite or its subsystems on a permanent basis.

r Class II: major nonrepairable failure that affects the operation of a satellite or its subsystems on a permanent basis.

r Class I: subsystem failure causing satellite retirements. This effectively means the total failure of the satellite due to a (dramatic) subsystem failure. Class IV and Class III events are lumped together hereafter, first because both classes of events have a minor impact on system or subsystem functionality, and second because the database does not contain information about the presence of subsystem redundancy, or lack thereof, on board spacecraft. The absence of this information precludes meaningful interpretation of the results of the separate statistical analysis of Class IV and Class III events. All the anomalies and failure events experienced by the spacecraft in the sample were collected, and their distribution across the different classes is shown in Figure 6.1. Partial failures of different severity constitute a significant portion of anomalous events experienced by spacecraft on orbit, and as such their analysis provides additional and important pieces of information in the understanding of spacecraft and subsystem failure behavior and propensity. Several observations can be made based on Figure 6.1:

r Complete spacecraft failures (Class I) represent a relatively small part of the problem events that occur on orbit: roughly 13% of these events result in the loss of the spacecraft; the remaining (majority of) events represent partial failures of minor or major severity.

r About 46% of the problems experienced by the spacecraft in the sample are major failure events (Class II), meaning that the anomaly resulted in a permanent Total: 773 Class I, 12.7% Class IV-III, 41.3%

Class II, 46.1%

Figure 6.1 Distribution of anomaly and failure events by severity for spacecraft successfully launched between January 1990 and October 2008.



86

February 17, 2011

10:8



and significant impairment of the satellite/satellite subsystem functionality (but not its total loss).

r Minor anomalies are also quite frequent (Class III and Class IV), and they represent 41% of all problem events experienced on orbit by the spacecraft in the sample. These numbers should not be overly interpreted beyond the important message that they convey: that is, that focusing solely on the reliability of spacecraft, defined as the probability of being in an operational (not total failure) state, misses an important part of spacecraft on-orbit degradation and failure behavior. Figure 6.1 can be construed as a justification of the need for multi-state failure analysis to be applied to spacecraft and their subsystems, the analysis of which is conducted in the following chapters and for which the ground is laid in this chapter. Section 6.3 explores the distribution of anomalies and failure events by subsystem, and Section 6.4 examines the time to anomaly and failure (of different severity) of these subsystems.

6.3

Distribution of anomalies and failure events by subsystem

The numbers of anomalies by class of failure events and per subsystem are given in Table 6.1. To help with the reading and visualization of these results, Figure 6.2 and Figure 6.3 contain two different breakdowns of the information in this table.

Table 6.1 Number of anomalies and failures by spacecraft subsystem. Subsystem Gyro Thruster Beam CP Mechanisms Payload Battery ED SAD SAO TTC Unknown All subsystems

Class IV–III anomalies

Class II anomalies

Class I failures

All anomalies and failures

86 29 4 24 8 99 6 7 2 22 32 0 319

35 55 17 1 9 93 16 10 6 77 23 14 356

16 14 0 5 7 3 8 9 2 9 19 6 98

137 98 21 30 24 195 30 26 10 108 74 20 773



February 17, 2011

10:8



87

Percentage of the 773 anomalies and failures 0%

5%

10%

15%

20%

25%

30%

Payload Gyro SAO Thruster TTC CP Battery ED Mechanisms Beam Unknown SAD 0

20

40

60 80 100 120 140 160 180 Number of anomalies and failures

200

220

240

Figure 6.2 Subsystem distribution of all 773 anomaly and failure events.

Figure 6.2 displays all the anomalies and failures per subsystem, regardless of the severity information (the right column in Table 6.1, in absolute numbers and percentages). The result is a preliminary indication of the likelihood of (propensity) a failure event of each subsystem, without the consequent information of such an event. Figure 6.2 shows for example that of all the failure events recorded in the population of spacecraft analyzed here (773 in total), 25.2% are due to the Payload, 17.7% to the Gyro, and 12.7% to the Thruster subsystems. The electrical power subsystem (EPS), which comprises the Battery, ED, and the SAD and SAO, accounts for 22.5% (174) of all failure events experienced on orbit. Figure 6.2 identifies five spacecraft subsystems as the leading sources of spacecraft failure events, namely the Payload, Gyro, SAO, Thruster, and TTC subsystems. About 80% of all anomalies and failures occurs in these subsystems. Accordingly, it can be tentatively advanced that improvements to these subsystems, through more extensive testing prior to launch, better design or parts selection, or redundancy, would significantly decrease the likelihood of spacecraft experiencing failure events. The failure severity information discussed next provides additional details to this statement. Figure 6.3 shows the contribution per subsystem to each of the three classes of failure events. Figure 6.3 reads as follows. For example, of all the minor failure events (Class IV–III) experienced on orbit in the population of spacecraft analyzed here, 31.0%

0

20 40 60 80 100 Number of Class IV-III anomalies

SAD Beam

SAD CP

Beam

SAD

Unknown

0

5

10 15 20 25 30 Number of Class I failures

10:8

Figure 6.3 Subsystem repartition of classes IV–III, II and I events.

CP Payload

Mechanisms

Battery

20 40 60 80 100 120 Number of Class II anomalies

Unknown

ED

ED

0

Mechanisms

Battery Unknown

Mechanisms

5% 10% 15% 20% 25% 30% 35%

Percentage of the 98 Class I failures

February 17, 2011

Battery

ED

TTC Beam

Gyro

CP

SAO

Thruster

TTC

Thruster

SAO

Thruster

SAO

TTC Gyro

Payload

Gyro

0%

Payload

5% 10% 15% 20% 25% 30% 35%

Percentage of the 356 Class II anomalies

Percentage of the 319 Class IV-III anomalies 0% 5% 10% 15% 20% 25% 30% 35%

Class I failures


0%

Class II anomalies

88

Class IV–III anomalies





February 17, 2011

10:8



89

were due to the Payload, 27.0% were due to the Gyro, and 10.0% to the TTC subsystems. Similarly the middle panel in Figure 6.3 identifies the subsystems that contribute the most to major anomalies on orbit, as the Payload (26.1%), the SAO (21.6%), and the Thruster (15.4%) subsystems. And finally, the right panel in Figure 6.3 identifies the leading subsystems whose failures lead to the complete loss of the spacecraft, as the TTC (19.4%), the Gyro (16.3%), and the Thruster (14.3%) subsystems. The electrical power subsystem (EPS), which, as noted previously, comprises the Battery, ED, and the SAD and SAO, has the lion’s share, 28.6%, of all failure events that lead to the loss of the spacecraft. Appendix B explores in further detail spacecraft EPS anomalies and failures. The contribution of Figure 6.3 is particularly revealing in the case of the Payload when compared with the previous results in Figure 6.2. As noted previously, the Payload is responsible for most of all failure events recorded on orbit (25.2%). Figure 6.3 indicates that the Payload experiences primarily minor and major anomalies, but rarely catastrophic failures that lead to the complete loss of the spacecraft: only 3% of Class I failures are due to the Payload. Conversely, while the TTC subsystem for example is the fifth largest contributor to on-orbit failure events, as seen in Figure 6.2, it is the lead contributor (19.4%) to catastrophic events leading to the complete loss of the spacecraft. These results better inform or help readjust the priorities for subsystem improvements. For example, Figure 6.3 identifies the primary spacecraft reliability degraders (causing Class I failures) as the TTC, Gyro, and Thruster subsystems. Accordingly, it can be advanced that improvements to these three subsystems, through more extensive testing prior to launch, better design or parts selection, or redundancy, would significantly improve spacecraft reliability. In addition, Figure 6.3 identifies the Payload, SAO, and Thruster subsystems as the primary sources of major anomalies on orbit. In the absence of resource constraints, all spacecraft subsystems would benefit from improvements. In the more realistic case of resource constraints, Figure 6.3 helps prioritize the target subsystems for improvement; the following is the authors’ proposed “Most Wanted” list of culprit subsystems in need of both careful analysis (of their degradation mode/mechanisms and physics of failure) and improvement: 1. Gyro. 2. TTC. 3. Thruster. 4. SAO. 5. Payload. The subsystems in a second-tier “Most Wanted” list for improvement would be: i. Battery. ii. ED. iii. Mechanisms. iv. CP.



90

February 17, 2011

10:8


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES Class I

Class II

Class IV-III

ED Mechanisms Battery TTC SAD CP Thruster Gyro SAO Payload Beam 0%

10%

20%

30%

40%

50%

60%

70%

80%

Severity distribution of anomalies and failures

90%

100%

0

50

100

150

200

Number of anomalies and failures

Figure 6.4 Severity distribution of anomalies and failures among spacecraft subsystems. Figure 6.4 provides another perspective on the degradation and failures experienced by spacecraft subsystems – another way of displaying the data in Table 6.1. The figure shows the distribution of classes of failures per subsystem. Consider for example the ED subsystem, shown at the top in Figure 6.4. Its failure events are distributed as follows: 34.6% are Class I failures, 38.5% are Class II anomalies, and 26.9% are Class IV–III anomalies. The subsystems in Figure 6.4 are ranked from top to bottom according to decreasing percentage of Class I failures. The bar-plot in the right panel displays the total number of anomalies and failures experienced by the subsystems. Figure 6.4 shows for example which subsystems, when they fail, are more likely to fail “hard,” resulting in major degradation or total loss of the spacecraft (e.g., ED, Mechanisms, Battery, SAD, even though these subsystems do not fail often; when they do, they tend to “break” the spacecraft). The following section adds a time dimension to this exploratory data analysis by examining the time to anomaly and failure (of different severity) of spacecraft subsystems.

6.4

Time to anomaly and failure of spacecraft subsystems

For each subsystem, the time to anomaly and failure event of different severity is collected and displayed in a box-plot format (see for example Figure 6.5). A boxplot reads as follows: the edges of the box are determined by the first quartile (25th percentile) and the third quartile (75th percentile) of the random variable considered (here the time to a failure event). The line within the box corresponds to the median value of the variable, and the whiskers outside the box represent the minimum and



February 17, 2011

10:8



91

maximum values. The diamond corresponds to the mean time to a failure event. In addition, associated with each box-plot is a bar-plot summarizing for each subsystem its distribution of failure events per class of failure. This allows a clear visualization of the prevalence of classes of failures, if any, for each subsystem, as will be discussed shortly. Note that the time to anomaly and failure events analyzed are actual times recorded in the database. As such, the statistics displayed here (e.g., mean and median) are conditional values: that is, for spacecraft that have experienced failure events, these are their representative statistics. As noted in the Introduction, this chapter is an exploratory data analysis, and it brings no formal statistical tools to bear on the data (e.g., no censoring is accounted for in what follows). This chapter, as stated earlier, is a turning point in this work and serves as a bridge between the formal binary-state reliability analysis in the previous chapters and the formal multi-state failure analysis in the following chapters. Figure 6.5 provides the time to anomaly and failure of the Gyro and the Thruster subsystems. These two subsystems are commonly subsumed under the broader appellation of the attitude and orbit control subsystem (AOCS) of the spacecraft. As

Gyro Total: 137 Class I

Class I

Class II

Class II

Class IV-III

Class IV-III

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time to Anomaly and Failure (years)

16

35

86

0

10

20 30 40 50 60 70 80 90 100 Number of anomalies and failures

Thruster Total: 98 Class I

Class I

Class II

Class II

Class IV-III

Class IV-III


14

55

29

0

10


Figure 6.5 Box-plots of time to anomaly and failure for the Gyro and Thruster subsystems.



92

February 17, 2011

10:8



seen previously, these two subsystems are major drivers of spacecraft anomalies and failures. The upper left quadrant in Figure 6.5 indicates that Gyro anomalies and failures are spread throughout the life of the spacecraft, as seen in the dispersion of the three times to anomalies or failure from year 1 to approximately year 14 on orbit. The median time to anomaly, which is more robust to outliers than the mean value, is roughly a year and a half for the minor anomalies (Class IV–III). In other words, 50% of all minor Gyro anomalies, which as seen in the upper–right quadrant of Figure 6.5 are significant and predominant, occur in the first year and a half after successful orbit insertion of the spacecraft. The median time to a major Gyro anomaly is approximately 3 years, and the time to complete failure is roughly 51 /2 years. It is interesting to note that all first quartiles of these three times to anomaly and failure events are less than 1 year. In other words, 25% of all anomalies and failures of the Gyro subsystem are concentrated in, or occur before, the first year on orbit. This observation has important implications as it suggests a clear propensity to infant mortality, and infant anomaly, of the Gyro subsystem, and as such a clear opportunity for satellite manufacturers to weed out these failures through improved testing procedures prior to launch. Finally it should be pointed out that the Gyro subsystem exhibits a progression in time and propensity between the anomaly and failure events of different classes: the minor anomalies occur more often and on average earlier than major anomalies; and the major anomalies occur more often and on average earlier than catastrophic failures. This behavior is likely indicative of distinctive failure mechanisms and design features of the Gyro subsystem (probably wear-out failures), in contrast with several other subsystems, for example, the Thruster subsystem as shown in the lower part of Figure 6.5, and discussed next. Note first that the Thruster, unlike the Gyro, subsystem is more likely to experience a major anomaly (56% of Class II failure events) than a minor anomaly. More importantly, the lower left quadrant in Figure 6.5 shows not only that the first quartile of the three times to anomaly and failure events is less than 1 year, but also that the median times are less than 2 years. In other words, 50% of all anomaly and failure events of the Thruster occur in the first 2 years on orbit – and in the first year on orbit for Class I and Class IV–III failure events. This result indicates an even more pronounced propensity to infant mortality, and infant anomaly, of the Thruster subsystem, than the Gyro, and it indicates a need for improving quality control and perhaps tailoring burn-in procedures for the Thruster subsystem. This represents another opportunity for satellite manufacturers to weed out a bundle of early and numerous failure events. It is interesting to note that the times to a minor anomaly and total failure of the Thruster are less dispersed than the corresponding ones of the Gyro subsystem: they are confined to the first 6 years on orbit in the case of the Thruster, whereas minor anomalies and total failures of the Gyro continue to occur over 13 years. Similar observations can be made regarding the third quartiles of the times to anomaly and failure events of both subsystems. This difference in failure behavior may be indicative of an increased robustness of the Thruster subsystem with time compared to the Gyro, especially if it can survive the first 6 years on orbit (more than 86% of all Thruster



February 17, 2011

10:8



93

failure events would had occurred by then), or a distinct propensity of the Gyro to wear-out failure events, in addition to its infant mortality, and infant anomaly, failures. The times to anomaly and failure events of the remaining subsystems listed in Section 6.2 are provided next. Similar observations and interpretations of the results can be made as for the preceding ones, in particular with respect to early failure events on orbit for all the subsystems. The Battery and ED subsystems, as noted previously, have a tendency to fail “hard”: over 70% of their anomaly and failure events are major anomalies (Class II) or catastrophic failures (Class I). Their median times to minor or major anomalies, shown in Figure 6.6, are confined to the first year on orbit. The time to complete failure of the Battery is distinctive in that its first quartile occurs after the third quartile of the time to major anomaly, and that complete failures continue to occur past 14 years on orbit (with a third quartile past 9 years). This may reflect two or more distinct failure modes in spacecraft batteries, and it represents from an actuarial perspective both infant anomaly and mortality, and wear-out failures. Anomaly and failure events of the SAO subsystem are mainly major anomalies (Class II). Figure 6.7 shows that the dispersion of the three times to anomaly and Battery Total: 30 Class I

Class I

Class II

Class II

Class IV-III

Class IV-III


8

16

6

0

10


ED Total: 26 Class I

Class I

9

Class II

Class II

10

Class IV-III

Class IV-III


7

0

10


Figure 6.6 Box-plots of the time to anomaly and failure of the Battery and ED subsystems.



94

February 17, 2011

10:8



SAD Total: 10 Class I

No box-plot is provided, as the SAD intervenes at the very beginning of the life of the spacecraft

2

Class II

6

Class IV-III

2

0

10


SAO Total: 108 Class I

Class I

Class II

Class II

Class IV-III

Class IV-III


9

77

22

0

10


Figure 6.7 Box-plots of time to anomaly and failure of the solar array (SAD and SAO).

failure events is relatively large and extends to 9 years or beyond, even though the three median times to anomaly and failure event are less than 2 years. This again indicates a clear propensity to infant mortality, and infant anomaly, of the SAO subsystem, and the need to improve the solar panels’ design, testing, and burn-in procedures. Figure 6.8 shows the time to anomaly and failure of the TTC subsystem.1 Recall that the TTC is a major reliability degrader of spacecraft. Similar observations to the previous ones can be made, in particular with respect to the three median times to anomaly and failure events being less than 2 years, thus representing an opportunity for satellite manufacturers to weed out these early anomalies and failure through improved burn-in procedures. Note also the locations of the first quartiles (less than 2 years) and the third quartiles (approximately 3, 4.5, and 5 years for the time to minor anomalies, major anomalies, and total failures respectively).

1 These

results exclude the endemic failures of the TTC subsystem of the GLOBALSTAR fleet (47 Class II failures).



February 17, 2011

10:8



95

Total: 74

Class I

Class I

Class II

Class II

Class IV-III

Class IV-III

19

23

32

0


10


Figure 6.8 Box-plots of time to anomaly and failure of the TTC subsystem. In the following, the box-plots of the time to anomaly and failure of the Payload (Figure 6.9), the Mechanisms (Figure 6.10), the CP (Figure 6.11), and the Beam (Figure 6.12) subsystems are provided. They invite similar observations and interpretations to the ones provided previously, and as a result they are not commented upon to avoid undue redundancy. Total: 195 Class I

Class I

Class II

Class II

Class IV-III

Class IV-III

0

1

2

3 4 5 6 7 8 9 10 11 12 13 14 15 16 Time to Anomaly and Failure (years)

3

93

99

0

10


90

100

Figure 6.9 Box-plots of time to anomaly and failure of the Payload subsystem. Total: 24 Class I

Class I

Class II

Class II

9

Class IV-III

Class IV-III

8

0

1

2

3 4 5 6 7 8 9 10 11 12 13 14 15 Time to Anomaly and Failure (years)

7

0

10


90

100

Figure 6.10 Box-plots of time to anomaly and failure of the Mechanisms subsystem.



96

February 17, 2011

10:8


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES Total: 30

Class I

Class I

Class II

Class II

Class IV-III

Class IV-III

0

1

2

3

4

5

6

7

8

5

1

24

0

9 10 11 12 13 14 15

10

Time to Anomaly and Failure (years)

20

30

40

50

60

70

80

90

100


Figure 6.11 Box-plots of time to anomaly and failure of the CP subsystem. Total: 21 Class I

Class I

Class II

Class II

Class IV-III

Class IV-III

0

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Time to Anomaly and Failure (years)

0

17

4

0

10

20

30

40

50

60

70

80

90

100


Figure 6.12 Box-plots of time to anomaly and failure for the Beam subsystem. The distinctive feature of the three times to anomaly and failure events of the Mechanisms subsystem, which is not seen for the previous subsystems, is a narrow dispersion with all classes of anomaly and failure events occurring approximately before the third year on orbit (and before the second year on orbit for the major anomalies and complete failures). The CP subsystem, shown in Figure 6.11, exhibits a particular catastrophic failure behavior: its Class I failures are significantly less preponderant than minor anomalies, but they occur more often later on orbit. The first quartile of the time to complete failure is past 4 years, and its median is around 7 years, compared to roughly 2 years and 3 years for the similar statistics of the time to minor anomaly. This is likely the result of redundancy in the design of the CP subsystem, a design strategy that seems to be successful, given the relatively small contribution of the CP to major and catastrophic spacecraft failures, as seen in Figure 6.3 (unlike the Gyro subsystem for example). The following chapter explores more formally the multi-state failure behavior of spacecraft subsystems.



February 18, 2011

7:42


7

Multi-state failure analysis of spacecraft subsystems1 7.1

Introduction

In the previous chapters, nonparametric reliability analyses and Weibull fits of spacecraft and spacecraft subsystems were conducted. The approach for investigating on-orbit failures consisted up to Chapter 5 of two phases. The first one tackled spacecraft reliability, confined to the system-level failures, and progressed from a general on-orbit failure behavior (Chapter 2 and Chapter 3) to more specialized reliability analysis of spacecraft by orbit and mass categories (Chapter 4). The second phase delved, beyond the system-level, into subsystem failures and derived reliability results (nonparametric and Weibull models) for all spacecraft subsystems (Chapter 5). In both phases, however, the statistical analysis and modeling were confined to the binary concept of reliability, in which an item is considered to be in either an operational or failed state. In this chapter, the investigation of failures of spacecraft subsystems is extended in a new direction beyond the binary concept of reliability to the analysis of anomalies and multi-state failures, or failure events of different severity (Figure 7.1). Multi-state failure analysis introduces “degraded states” or partial failures, and thus provides more insights through finer resolution into the degradation behavior of an item and its progression toward complete failure. For example, the database used for the statistical analysis in the present work identifies four classes of anomaly and failure for each satellite subsystem: three degraded states; and one failed state (complete 1 This chapter is based on an article written by the authors and published in Reliability Engineering and System Safety (Castet and Saleh, 2010b).




98

February 18, 2011

7:42



SATELLITE

SATELLITE Subsystem 1

O

Subsystem i

O

O

O

... F

Spacecraft reliability * * Work done in Chapters 2 and 3

F

... F

Multi-state failure analysis of spacecraft subsystems

** Work done in Chapter 5

SATELLITE Subsystem i

O

O – Operational state

O D1

...

D1 – Minor degraded state

D1

...

D2 F

Subsystem n

O D1

F – Failed state

D2 – Major degraded state

F

Spacecraft subsystems reliability **

Subsystem 1 Legend:

Subsystem n

D2 F

D2 F

Figure 7.1 Progression in the statistical analysis of spacecraft and spacecraft subsystem failures. failure). These four states are identified and discussed in Section 7.2. By conducting multi-state failure analysis, additional details and insight can contribute more to an understanding of on-orbit failures and anomalies than what is possible with traditional reliability analysis, as will be shown hereafter. This chapter builds on Chapter 6, where time to anomaly of spacecraft subsystems was explored, and brings formal statistical methods to investigate in more detail the functionality degradation of spacecraft subsystems and probability of transitioning between different states. Details about the database used in this work can be found in Chapter 2. The subsystems considered here were introduced in Chapter 5. For convenience, the list is repeated below: 1. Gyro/sensor/reaction wheel (hereafter referred to as Gyro). 2. Thruster/fuel (Thruster). 3. Beam/antenna operation/deployment (Beam). 4. Control processor (CP). 5. Mechanisms/structures/thermal (Mechanisms). 6. Payload instrument/amplifier/on-board data/computer/transponder (Payload). 7. Battery/cell (Battery).



February 18, 2011

7:42


MULTI-STATE FAILURE ANALYSIS OF SPACECRAFT SUBSYSTEMS

99

8. Electrical distribution (ED). 9. Solar array deployment (SAD). 10. Solar array operating (SAO). 11. Telemetry, tracking, and command (TTC). When the culprit subsystem that led to the failure of the spacecraft cannot be identified, the failure of the spacecraft is ascribed to an “Unknown” category in the database. The remainder of this chapter is organized as follows. In Section 7.2, different classes of failures are introduced along with the state transition diagram, which serves as the basis for the statistical multi-state failure analysis that follows. In Section 7.3, for each spacecraft subsystem, nonparametric analyses of probabilities of transitioning between different states are calculated along with their confidence intervals. In Section 7.4, parametric analysis and Weibull fits are derived for these probabilities using the MLE procedure. Finally in Section 7.5, a comparative analysis of the reliability results versus multi-state failure analyses of three spacecraft subsystems is provided as a way to illustrate the importance and additional insights that derive from such an analysis compared to a traditional binary reliability analysis. The three subsystems analyzed in Section 7.5 are the previously identified major culprits driving spacecraft unreliability (Chapter 5), and that would benefit most from reliability improvements, namely, the Thruster, TTC, and the Gyro subsystems.

7.2

Setting the stage: multi-state failure analysis and the state transition diagram

As noted previously, the novel contribution in this chapter is the extension of the statistical analysis of spacecraft subsystem failures beyond the binary understanding of reliability to the analysis of subsystem anomalies and multi-state failures. The statistical analysis is enabled by, and confined to, the classes of (partial) failures identified in the database. These classes of failures, as defined in the database, are listed next in increasing order of failure severity:




r Class I: subsystem failure causing satellite retirement. This effectively means the total failure of the satellite due to a (dramatic) subsystem failure.



100

February 18, 2011

7:42



Fully operational

S4 S3

S2 Class I Total failure

Class IV–Class III Minor anomaly/degradation

Class II Major anomaly/degradation

S1

Figure 7.2 Multi-state and transition diagram for spacecraft subsystem failure behavior.

Building on these classes of events, the following four states are introduced, numbered from 4 to 1 in increasing order of failure severity or functionality degradation:

r State 4: fully operational. r State 3: Class IV and Class III partial failure (minor anomaly/degradation). r State 2: Class II partial failure (major anomaly/degradation). r State 1: Class I total failure. Class IV and Class III events have been lumped into one state, first because both classes of events have a minor impact on the system or subsystem functionality, and second because the database does not contain information about the presence of subsystem redundancy, or lack thereof, on board spacecraft. The absence of this information precludes meaningful interpretation of the results of the separate statistical analysis of Class IV and Class III events. The failure state diagram for each subsystem is shown in Figure 7.2. Once the probabilities of transitioning between each state for each subsystem are calculated in Section 7.3, the statistical multi-state failure behavior of the satellite and its subsystems can be modeled and simulated using Markov chains or stochastic Petri nets, as will be discussed in Sections 7.3 and 7.4. State 1 (Class I failure) is referred to in stochastic modeling as an absorbing state: it cannot be recovered from, and as such no outbound transitions emanate from it. No transitions are shown in Figure 7.2 from a partially failed state toward a higher functional state (i.e., no directed arc from Si to S(i + 1) for i = 1). In reality, few transitions (3.6%) in the database occur in this “healing” direction. Incidentally, the argument in support of on-orbit servicing can be made in relation to these “healing” state transitions. This subject, however, is beyond the scope of the present work (see Saleh et al. (2003) for details about on-orbit servicing).



February 18, 2011

7:42



101

Consider the following notations: Tij : transition between the state i and state j; Pij : conditional probability2 of transitioning from state i to state j. For example, the transition for a subsystem from a fully operational state (S4) to a major anomaly (S2) is labeled T 42 , and the probability of transitioning between these two states is P42 . The transition probabilities between the states are obtained from a statistical analysis of the anomaly and failure data. The procedure and the calculations are described in the following section. The reader interested in details about multi-state systems in general is referred to Lisnianski and Levitin (2003).

7.3

Nonparametric analyses of spacecraft subsystems’ multi-state failures

In this section, the failure and anomaly data from all the states previously defined is used to compute the probability of transitioning from one state to another for all the spacecraft subsystems referenced in Section 7.1. The following data is collected for each subsystem and each state transition (i,j): (1) the date of arrival in state i; (2) the date of leaving state i to state j, if this transition occurred; and (3) the “censored time” if the state transition (i,j) did not occur. Particular attention is required in handling censoring. In addition, beyond the procedure for handling right-censored data in the binary case of reliability analysis described in Chapter 2, multi-state failures introduce an additional subtlety in the definition of censored data and its handling. These issues are discussed in the following subsection.

7.3.1

Censored data and the Kaplan–Meier estimator

The data set, as was the case previously, is right censored (Type 4 censoring) with staggered entry. This means that:

r The subsystems in the sample are activated (arrival date in state i or launch date for i = 4) at different points in time, but all these activation times are known.

r Departure dates from state i to state j are stochastic (and so is censoring). r Censoring occurs because a spacecraft is retired from the sample before Tij occurs or because the end of the observation window is reached (October 2008) without the subsystem experiencing the transition Tij . In addition, in multi-state failure analysis, when studying Tij for a given subsystem, censoring also occurs when the subsystem transitions to a state k different from state j. In this case, Tik 2 Additional

precision with respect to this term will be discussed in Section 7.3.



102

February 18, 2011

7:42



with k = j is considered censoring for the calculation of Tij . For example, when studying T 43 , that is, the transition of a subsystem from the fully operational state to the minor anomaly/degradation state, T 43 is censored by T 42 and T 41 , the transitions to state 2 and state 1 (major anomaly/degradation and total failure). The nonparametric reliability procedure for censored failure data can be found in Chapter 2, where two states of an item are considered: operational (O) and failed (F). The probability of remaining in the operational state up to a time t is the reliability function R(t) of the item. The probability of having transitioned to a failed state from the operational state by the time t is P(t) = 1 − R(t). Thus, by applying the Kaplan–Meier estimator, the estimate Pˆ of the probability of failure P is given by

ˆ = 1 − R(t) ˆ =1− P(t)

pˆ i .

(7.1)


In this chapter, the Kaplan–Meier estimator is adapted to estimate the conditional probability Pij of transitioning from state i to state j, that is, the estimator is adapted to handle multi-state failures with their distinct censoring. To illustrate this point, consider the transition diagram shown in Figure 7.3. The following focuses on estimating the probability of transitioning from state i to state j, Pij . This in effect is a conditional probability, which means that if the item is in state i, it is Pij likely to have transitioned to state j by the time t. Recall that censoring in the binary reliability analysis implies that an item has been removed from observation (for various reasons) prior to the occurrence of failure. In multi-state failure analysis, any transition to another state than the one of interest, in the example from state i to state j, is also considered censoring. For example, in Figure 7.3 the transitions from state i to state r or s (r = j and s = j) are considered censoring for the calculation of Pij . Thus, similar calculations to those in Equation (2.10) can be used to adapt the Kaplan–Meier estimator to Pij . The estimate Pîj of Pij is written as

Pîj (t) = 1 −

pˆ ij, k

all k such that tij(k) ≤t

i

Pij j

r

s

Figure 7.3 Censoring of Pij .

(7.2)



February 18, 2011

7:42



103

where tij(k) is the time to the kth departure from state i to state j (arranged in ascending order), pˆ ij, k =

n ij, k − 1 , n ij, k

(7.3)

and n ij,k = number of units in state i right before tij(k) = n − number of censored units right before tij(k) − number of units having transitioned to state j right before tij(k) . Should there be ties in the transition times tij(k) , say mij,k units transitioning at exactly tij(k) , then (7.3) is replaced by pˆ ij, k =

7.3.2

n ij, k − m ij, k . n ij, k

(7.4)


The Kaplan–Meier estimator (Equation (2.10)) provides a maximum likelihood estiˆ i ). Similarly, in mate of reliability but does not inform about the dispersion around R(t multi-state failure analysis, (7.2) does not inform about the dispersion around Pîj (t). This dispersion is captured by the variance or standard deviation of the estimator, which is then used to derive the upper and lower bounds for, say, a 95% confidence interval, for example, the 95% likelihood that the actual conditional probability Pij will fall between the two calculated bounds, with (7.2) providing the most likely estimate. The variance of the estimator, provided by Greenwood’s formula (2.12), is adapted to multi-state failure analysis as follows: 2 vâr Pij (tij(k) ) ≡ σ 2 (tij(k) ) = 1 − Pîj (tij(k) ) · l≤k

m ij,l n ij,l (n ij,l − m ij,l )

(7.5)

and the 95% confidence interval becomes Pij,95% (tij(k) ) = Pîj (tij(k) ) ± 1.96 · σ (tij(k) ).

7.3.3

(7.6)

Nonparametric estimations of the conditional probabilities of transitioning between states

With the background information provided in the previous subsection, the multi-state failure analysis of spacecraft subsystems can now be applied to the on-orbit anomaly and failure data of the 1584 spacecraft in the sample. Recall that these spacecraft were launched between January 1990 and October 2008. The data is treated with (7.2)–(7.6) to obtain the nonparametric estimations Pîj of Pij .


104

February 18, 2011

7:42


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES 0.07 0.06


0.05 Probability


0.04 0.03 0.02 0.01 0.00 0


Figure 7.4 Probability of transitioning Pˆ42 for the Thruster subsystem. It is worth pointing out that with 11 subsystems and 6 possible state transitions for each subsystem to calculate, there are 66 nonparametric probabilities to estimate (excluding the Unknown category). In addition, two additional calculations for each transition probability are required to estimate the 95% confidence interval. As a result, 198 nonparametric calculations are needed to fully characterize the multi-state failure behavior of the spacecraft in the database, given the number of subsystems and the classes of failures identified.3 This proliferation of transition probabilities is in effect one of the main difficulties in statistically handling multi-state failures compared to the simple situation of the binary reliability analysis, and is rightfully characterized as “dimension damnation” by Lisnianski and Levitin (2003). However, the insights that emerge from multi-state failure analysis are significantly worth this added complexity, as will be shown shortly. Figure 7.4 illustrates an example of the nonparametric calculations. Shown in the figure is Pˆ42 for the Thruster subsystem, that is, the estimated conditional transition probability from the fully operational state (state 4) to the major anomaly (state 2) for this subsystem, along with the 95% confidence interval. Figure 7.4 reads as follows. After 10 years on orbit, for example, the Thruster subsystem of a spacecraft is 4% likely to have transitioned to a major anomaly (state 2). More precisely, Pˆ42,thruster (t) = 0.0387 for 9.210 years (3364 days) ≤ t < 10.453 years (3818 days). 3 This

result can be generalized for the multi-state failure analysis of a system with n subsystems and k states. The total number of nonparametric probabilities to estimate is given by n · k · (k − 1). In addition, i . When confidence assuming no “healing” transitions, the number reduces to n · k · (k − 1) − i=k−1 i=1 intervals are also sought, these numbers are tripled.


February 18, 2011

7:42



105

P42

P43

0.045

0.11 Nonparametric estimation 95% confidence interval bounds

0.10 0.09

0.040 0.035 0.030

0.07

Probability

Probability

0.08 0.06 0.05 0.04

0.025 0.020 0.015

0.03

0.010

0.02

0.005

0.01 0.00 0


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years) P41

P32

0.018

0.35

0.016

0.30

0.014

0.25 Probability

Probability

0.012 0.010 0.008 0.006

0.20 0.15 0.10

0.004 0.002

0.05


0.00 0 1

2

3

4

5

P31

0.45

0.10

0.40 0.35

0.08

0.30 Probability

0.07 0.06 0.05 0.04

0.25 0.20 0.15

0.03

0.10

0.02

0.05

0.01 0.00 0

6 7 8 9 10 11 12 13 14 15 Time (years) P21

0.11 0.09

Probability


1

2

3

4

5

6 7 8 9 10 11 12 13 14 15 Time (years)

0.00 0 1

2

3

4

5

6 7 8 9 10 11 12 13 14 15 Time (years)

Figure 7.5 Probabilities of transitioning for the Gyro subsystem.

In addition, P42 of this subsystem will fall between 2.72% and 5.02%, its 95% confidence interval at t = 10 years, with Pˆ42 = 3.87% being the most likely estimate. Figure 7.5 shows the six transition probabilities of the Gyro subsystem. Consider the most likely estimate Pîj on each panel in Figure 7.5. For example, after 4 years on orbit, the Gyro subsystem is roughly 4.8% likely to have



106

February 18, 2011

7:42


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES P43

P42

0.035

0.06

0.030

0.05

0.025

Probability

Probability

0.04 0.020 0.015

0.02

0.010 0.005

0.03

0.01




P41 0.016 0.014

Probability

0.012 0.010 0.008 0.006 0.004 0.002 0.000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure 7.6 Probabilities of transitioning from the fully operational state to all other states for the Thruster subsystem.

transitioned from state 4 to state 3 (minor anomaly), 1.3% likely to have transitioned from state 4 to state 2 (major anomaly), and 0.3% likely to have transitioned from state 4 to state 1 (total failure). The probabilities of transitioning P41 , P31 , and P21 provide a finer resolution in the mechanisms leading to the total loss of the spacecraft, as opposed to the traditional reliability analyses that lump together these three transitions. Figure 7.6 and Figure 7.7 show the three transition probabilities of the Thruster and TTC subsystems from the fully operational state to the three other states. Several transitions between states for various subsystems are not present in the data set analyzed here. For example, for the Thruster subsystem no transition occurred on orbit between a minor anomaly (state 3) and a complete failure (state 1) in the data set. As a result, this transition is not subject to statistical analysis. Other transitions also do not occur in the data set, thus reducing the total number of transitions to 48 and with a total of 144 nonparametric calculations (excluding the Unknown category). The absent transitions can be seen in Tables 7.1 and 7.2 below,


February 18, 2011

7:42


MULTI-STATE FAILURE ANALYSIS OF SPACECRAFT SUBSYSTEMS P43

107

P42 0.035

0.04

0.030 0.03

0.025 Probability

Probability

0.02

Nonparametric estimation 95% confidence interval bounds 0.00 0

0.020 0.015 0.010

0.01

0.005



P41 0.018 0.016 0.014 0.012 Probability


0.010 0.008 0.006 0.004 0.002 0.000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure 7.7 Probabilities of transitioning from the fully operational state to all other states for the TTC subsystem.

noted as “NA.” Although these 144 calculations are not shown here for convenience, the parametric fits for all these transition probabilities are provided in the next section. Limitation: In this subsection, it was implicitly assumed that subsystem failures are independent. In reality, some subsystems may have dependent anomalies and failures, for example, the thermal and power subsystems. Unfortunately, the information available in the database, and sometimes in the satellite operator’s incident report itself, does not explicitly address failure dependence. For example, a spacecraft Class I failure is ascribed to only one subsystem, and a partial failure of a subsystem has its timing and severity recorded. As noted previously, the statistical analysis in this work is enabled by and confined to the data available. As a result, common-cause and dependent anomalies and failures of spacecraft subsystems cannot be clearly identified and statistically analyzed. Such analyses, however, are important and constitute fruitful avenues for future research when the requisite data is collected.



February 18, 2011

7:42


108


7.4

Parametric analyses of spacecraft subsystems’ multi-state failures

As discussed in previous chapters, nonparametric analysis has advantages and limitations, the latter primarily related to the lack of portability and compactness in expressing results (see Chapter 3 for details). Building on previous results, it will be shown in this section that the Weibull distribution is a proper fit for the nonparametric probabilities of transitioning from state i to state j for most subsystems and state transitions. Since the interest herein is the cumulative failure likelihood (the transition to a degraded state), the shape and scale parameter of the following distribution (not the reliability function or the likelihood of remaining in an operational state) are calculated: t β Pij (t) = 1 − exp − . θ

(7.7)

Several methods can be used to determine a parametric model given failure and anomaly data, such as the graphical estimation or MLE procedures. In this chapter, the more robust MLE technique is adopted to estimate the shape and scale parameter of all the transition probabilities. The reader is referred to Chapter 3 for details about the MLE and its analytic derivation in the case of the Weibull distribution.

7.4.1

MLE Weibull fit of the nonparametric estimates Pîj

The MLE procedure is now applied to calculate the Weibull parameters of the probabilities of transition Pîj for all the spacecraft subsystems. The results are provided in Table 7.1 and Table 7.2. The transitions between states that are not present in the database cannot be subject to statistical analysis and are noted in these tables as “NA.” The information in Table 7.1 reads as follows. Consider for example the Gyro subsystem. Its MLE-fitted Weibull distribution for the probability of transitioning from state 4 to state 3, Pˆ43 , is given by

0.4731 t P43,gyro (t) = 1 − exp − . 2758

(7.8)

Similarly, the Thruster subsystem has its MLE-fitted Weibull distribution for the probability of transitioning from state 4 to state 2, Pˆ42 , given by

0.4763 t P42,thruster (t) = 1 − exp − . 8591

(7.9)



February 18, 2011

7:42



109

Table 7.1 Weibull parameters for the spacecraft subsystems Pij (β is dimensionless, θ is given in years). Gyro

Thruster

Pij

β

θ

Pij

β

P43 P42 P41 P32 P31 P21

0.4731 0.3685 0.5635 1.1950 0.7551 0.4653

2758 336 231 65 547 33 546 134

P43 P42 P41 P32 P31 P21

0.3827 0.4763 0.3114 0.6052

CP Pij P43 P42 P41 P32 P31 P21

β

Beam θ

171 879 8591 29 975 357 46 NA 0.2632 589 300

Pij P43 P42 P41 P32 P31 P21

Mechanisms θ

0.6585 3562 NA NA 0.5487 1056 0.7231 45 1∗

Pij P43 P42 P41 P32 P31 P21

β

β

θ

0.0019∗ 0.2468 436 409 190 NA NA NA NA Payload

θ

0.3840 4 952 368 0.0060∗ 0.3572 19 794 952 NA NA NA

Pij

β

P43 P42 P41 P32 P31 P21

0.4474 0.4691 0.6701 0.6647

θ

4065 3170 119 171 38 NA 0.2513 169 439 610

∗ Due to the constant form of the nonparametric curve, a Weibull fit is not meaningful in these cases. The values are the probabilities of transitioning over 15 years.

Figure 7.8 shows the nonparametric curves (with the 95% confidence interval) for the Pˆ43 of the Gyro subsystem, and the Pˆ42 of the Thruster subsystem, superimposed on their respective MLE Weibull fits as provided by Equations (7.8) and (7.9). Figure 7.8 provides visual confirmation that the Weibull distributions with the MLE parameters given in Table 5.1 are good fits for the Pˆ43 of the Gyro subsystem and the Pˆ42 of the Thruster subsystem. Similar results are obtained for the other probabilities of transitioning of the spacecraft subsystems using the Weibull parameters given in Table 7.1 and Table 7.2. Further confirmation is provided in the next subsection.

7.4.2

Testing the parametric models of the multi-state failure analysis

Given the relative complexity of subsystem models, several tests were devised to verify that the parametric models were properly derived, and that they reflected actual on-orbit data, as will be discussed shortly. In essence, these tests provided partial verification and validation of the subsystem models (IEEE Std 1012-2004). To proceed, two independent calculations were conducted for each subsystem of being in state 1, P1,sub (the complement of the reliability of each subsystem). The first method of calculation is based directly on the failure information in the database and is



110

February 18, 2011

7:42



Table 7.2 Weibull parameters for the spacecraft subsystems Pij (β is dimensionless, θ is given in years). Battery Pij

β

P43 P42 P41 P32 P31 P21

0.3855 0.4134 0.9239

ED θ

9 946 825 357 357 4431 NA NA 0.2353 1936

Pij

β

P43 P42 P41 P32 P31 P21

0.3663 0.3526 0.5215 1.1329

SAD θ

13 753 674 11 893 973 144 569 38 NA 0.4618 376

Pij

Pij

β

θ

Pij

β

P43 P42 P41 P32 P31 P21

0.3216 0.4724 0.2527 0.7268 0.5935 0.4307

3 237 079 4313 3.45×1010 16 646 4501

P43 P42 P41 P32 P31 P21

0.3668 0.5249 0.3098 0.2273

θ 0.0015∗ 0.0040∗ 0.0013∗ NA NA NA

P43 P42 P41 P32 P31 P21

TTC4

SAO

β

Unknown θ

205 920 19 577 29 482 835 390 440 NA 0.3374 87

Pij P43 P42 P41 P32 P31 P21

β 0.3766 0.4020

θ NA 1 471 383 5 578 316 NA NA NA

∗ The SAD “is a one-shot subsystem” and a Weibull fit is not meaningful in this case. Thus these are the probabilities of transitioning over 15 years.

independent of the multi-state failure analysis. The nonparametric results derived by this method are discussed in Chapter 5 and are referred to herein as the “benchmark” results. The second method uses the multi-state Weibull models derived in this work and runs a Monte Carlo simulation of a stochastic Petri net (SPN) model for each subsystem. SPNs are timed automata used in this work to model the multi-state failure behavior of each subsystem (Cassandras and Lafortune, 2007); this approach corresponds to model the state transition diagram in Figure 7.2 with the additional information that the timing of the transitions in the SPN models are governed by the Weibull models in Table 7.1 and Table 7.2. An example SPN model of the Gyro spacecraft subsystem is provided in the appendix to this chapter, as modeled with the SPN@ software (Volovoi, 2004; 2006). Details about SPNs can be found in Chapter 8. With 1 million runs, the Monte Carlo simulation allows a number of statistics to be monitored in the SPN models for all subsystems, including residency times in each state and, most importantly for validation purposes, the probability of each subsystem of being in state 1. The results derived by this second approach are referred to as the “simulated” results. 4 These

results exclude the endemic failures of the TTC subsystem of the GLOBALSTAR fleet (47 Class II failures).


February 18, 2011

7:42



111

P42 - Thruster / Fuel

P43 - Gyro / Sensor / Reaction wheel 0.11

0.06

0.10 0.05

0.09 0.08

0.04 Probability

0.07 Probability


0.06 0.05 0.04

0.03

0.02

0.03 0.02 0.01 0.00 0

0.01 Nonparametric estimation MLE Weibull 95% confidence interval bounds

Nonparametric estimation MLE Weibull 95% confidence interval bounds 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

0.00 0


Figure 7.8 Examples of nonparametric probabilities of transitioning and Weibull fits. Next, the “simulated” results are compared to the “benchmark” results. In the multi-state failure models, the dynamics of the simulated P1,sub is contingent on a host of transition probabilities, and any error or lack of precision in one Weibull model of these transition probabilities will be propagated and is likely to be amplified through the simulation of the SPN model.5 As a result, if the “simulated” results do not follow the actual “benchmark” results, one can conclude that the multi-state failure analysis and parametric fits are (in whole or in part) flawed or incorrect. Conversely, if the “simulated” results properly follow the “benchmark” results, this would provide a good indication that the multi-state failure analysis and parametric fits are likely correct and exhaustive, or, more precisely, that the failure behavior derived from the multi-state failure analysis and parametric fits properly mimics or numerically reflects the actual total failure behavior of the subsystem considered. This approach constitutes a partial, but important, validation of the parametric fits of the multi-state failure analysis. Figure 7.9 presents the benchmark and simulated results of 1 – P1,sub for two spacecraft subsystems, the Gyro and the Thruster subsystems. Figure 7.9 provides visual confirmation that the simulated results properly reflect (numerically) the actual “benchmark” results. The simulated results exhibit good accuracy with a maximum error of 0.40 percentage point for the Gyro subsystem, and 0.24 percentage point for the Thruster subsystem (Table 7.3). 5 To illustrate this argument, consider a subsystem which transitions to complete failure (state 1) primarily through state 3. Miscalculation of the P43 of this subsystem will be directly propagated in the simulation of the SPN model and result in a misevaluation of the subsystem’s P1,sub .



112

February 18, 2011

7:42



1.000

1.000 0.998

Nonparametric (benchmark) results Multi-state failure simulation

0.995

0.990

Reliability

Reliability

0.996

0.985

0.992 0.990 0.988

0.980 Nonparametric (benchmark) results Multi-state failure simulation 0.975 0

0.994

0.986



Figure 7.9 “Benchmark” reliability results and the simulated results (1 − P1,sub ) for two spacecraft subsystems: Gyro (left) and Thruster (right). Table 7.3 Error between the “benchmark” and the simulated results for the Gyro and Thruster spacecraft subsystems. Maximum error Subsystem Gyro Thruster

Average error


0.13 0.10

Similarly accurate results are obtained for the remaining subsystems, as shown in Table 7.4. These results provide a good indication that the multi-state failure analysis and parametric models derived here are appropriate and exhaustive (i.e., no transitions Table 7.4 Error between the “benchmark” and the simulated results for remaining subsystems. Maximum error Subsystem Beam CP Mechanisms Payload Battery ED SAD SAO TTC

Average error

(percentage point) –– 0.16 0.21 0.08 0.66 0.18 –– 0.29 0.38

–– 0.06 0.07 0.03 0.14 0.06 –– 0.11 0.13



February 18, 2011

7:42



113

were overlooked). More precisely, computational models of multi-state failures of satellite subsystems numerically reflect the actual subsystems’ failure behavior with significant accuracy. It should be noted that the accuracy reflected in Figure 7.9, Table 7.3, and Table 7.4 was obtained with single Weibull distributions modeling most of the probabilities of transitioning between the states Pij of the multi-state failure model. More accurate models can be developed for Pij , if more accuracy is needed, by using for example mixture distributions as described in Chapter 3.

7.5

Comparative reliability and multi-state failure analysis of spacecraft subsystems

In this section, complete multi-state failure results are provided, resulting from SPN simulations, for three spacecraft subsystems: the Gyro subsystem, the Thruster subsystem, and the TTC subsystem. These three subsystems were chosen in part because they were identified in Chapter 5 as the major culprits driving spacecraft unreliability, and in Chapter 6 as major drivers or spacecraft anomalies as well. In addition, these three subsystems were chosen because their multi-state failure analysis clearly identifies key insights that cannot be captured by the traditional (binary) reliability analysis.

7.5.1

Gyro subsystem

The Gyro subsystem is a major contributor to spacecraft failures with 18% of total spacecraft failures due to this subsystem over a period of 10 years, and 20% over a period of 15 years (see Chapter 5 for additional details). Figure 7.10 shows the reliability curve along with the probability of being in state 4, that is, being fully operational. The reliability curve, or survivor function, represents the probability of the subsystem not being in the failed state 1. Figure 7.11 shows the different probabilities of being in degraded states, from state 1 to state 3. Figure 7.10 highlights an important distinction that is made in multi-state failure analysis but that cannot be captured by the traditional (binary) reliability analysis, namely, the distinction between being in a fully operational state and being in the nontotal failure state. To clarify this point, consider the following. The two curves in Figure 7.10 are separated by a distinct and growing gap, with roughly 9 percentage point difference at t = 15 years. The upper reliability curve indicates that the subsystem is 98% reliable after 15 years, that is, the subsystem is 98% likely to be operational (not broken), whereas the multi-state failure analysis (lower curve in Figure 7.10) indicates that the subsystem is only 89% likely to be fully operational after 15 years. The difference is not negligible and can have important consequences, the most important probably being that a 98% reliable subsystem after 15 years might not trigger any engineering action, whereas an 89% fully operational subsystem might prompt a careful analysis of the subsystem (partial) failure modes and support improvement efforts. The distinction between these two probabilities of a subsystem occupying different states (fully operational versus nontotal failure states) lies of course in the


114

February 18, 2011

7:42


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES 1.00

0.98

Probability

0.96

Reliability Operational (S4)

0.94

0.92

0.90

0.88 0


Figure 7.10 Reliability and probability of being fully operational for Gyro subsystem. 0.06

0.05

Minor (S3) Major (S2) Failed (S1)

0.04 Probability


0.03

0.02

0.01

0.00 0


Figure 7.11 Degraded states probabilities for Gyro subsystem.



February 18, 2011

7:42



115

partial failures that are introduced and probed by the multi-state failure analysis. The probabilities of occupying any one of the failure states over 15 years are shown in Figure 7.11. This figure reads as follows. At t = 10 years, for example, there is a 5.5% probability that the subsystem is in a minor anomaly state (S3), 2.4% that the subsystem is in a major anomaly state (S2) – these states and probabilities are not visible to the traditional reliability analysis – and 1.5% that the subsystem has totally failed (S1). This last probability is in effect the complement of the reliability of the subsystem (the failed curve in Figure 7.11 is the complement of the reliability curve in Figure 7.10). Additional information and trends can be gleaned from Figure 7.11. For example, the Gyro subsystem is most likely to be in a minor anomaly state (S3) than in any other failure state. The probability of being in a major anomaly state (S2) remains higher than the probability of being in the total failed state (S1) with roughly a 0.9 percentage point difference. And more importantly, a clear “infant degradation” trend is noticeable – in parallel with the concept of infant mortality in the traditional reliability analysis – for this subsystem as evidenced by the rapid increase in the probabilities of being in all the partial failure states during the early years. The results provided by multi-state failure analysis can thus be used to prompt further detailed investigation into the “physics of anomaly and failure” of particular spacecraft subsystems and guide technical efforts toward the identification of subsystem failure modes and their elimination.

7.5.2

Thruster subsystem

The Thruster subsystem is another major contributor to spacecraft failures, especially over the early years of spacecraft service life. For example, over the first 10 years on orbit, 13% of all spacecraft failures are due to the Thruster subsystem, and for the first year on orbit, 20% of all spacecraft failures are due to this subsystem (Chapter 5). Figure 7.12 shows the reliability curve and the probability of being in state 4, that is, the probability of being fully operational for the Thruster subsystem. And Figure 7.13 shows the different probabilities of being in degraded states, from state 1 to state 3. As with the previous subsystem, the probability of being fully operational is significantly lower that the reliability of the Thruster subsystem, as seen in Figure 7.12. A significant 7 percentage point difference exists between the reliability curve of this subsystem and its probability of being fully operational at t = 15 years. The most interesting feature of the multi-state failure analysis of this subsystem is the dynamics of the degraded states, especially the probability of being in the major anomaly state (S2), as shown in Figure 7.13. The probability of being in a minor anomaly is low (less than 2%), whereas the probability of being in a major anomaly state is significantly higher, continuously increasing over the years to eventually reach approximately 5% after 15 years. The rapid increase in the probability of transitioning to state 2 (major anomaly) in the early years, as seen in Figure 7.13, can be termed “infant severe degradation” of the Thruster subsystem, as the multi-state analog of the infant mortality concept in traditional reliability analysis.


116

February 18, 2011

7:42


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES 1.00 0.99 0.98

Probability

0.97 0.96 Reliability Operational (S4)

0.95 0.94 0.93 0.92 0.91 0.90 0


Figure 7.12 Reliability and probability of being fully operational for the Thruster subsystem. 0.06

0.05


0.04 Probability


0.03

0.02

0.01

0.00 0


Figure 7.13 Degraded states probabilities for the Thruster subsystem.


February 18, 2011

7:42



117

In summary, when the Thruster subsystem (partially) fails, it is likely to “fail hard,” that is, with a transition to a major anomaly/degradation state (S2). This constitutes a different failure behavior from the Gyro subsystem for example, which is more likely to be in a minor anomaly (S3) than in a major anomaly state (S2), as seen previously. The Thruster subsystem has previously been identified as one of the major culprits driving spacecraft failures (Chapter 5). The present multi-state failure analysis also shows that this subsystem experiences significant degradation in its functionality on orbit. This provides an additional indication that spacecraft manufacturers and equipment providers should focus their attention on improving the Thruster subsystem and more generally on subsystems that either drive spacecraft failures or have a high propensity for major degradation.

7.5.3

TTC subsystem

The TTC subsystem is another major contributor to spacecraft failures. For example, over the first 10 years on orbit, 19% of all spacecraft failures are due to the TTC subsystem (Chapter 5). Figure 7.14 shows the reliability curve and the probability of being in state 4, that is, the probability of being fully operational for the TTC subsystem. Figure 7.15 shows the different probabilities of being in degraded states, from state 1 to state 3. Similar observations can be made regarding these results as the ones previously discussed and will not be repeated here to avoid (excessive) redundancy.

1.00

0.99

0.98

0.97 Probability


0.96


0.95

0.94

0.93

0.92 0


Figure 7.14 Reliability and probability of being fully operational for the TTC subsystem.


118

February 18, 2011

7:42


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES 0.04 Minor (S3) Major (S2) Failed (S1) 0.03

Probability


0.02

0.01

0.00 0


Figure 7.15 Degraded states probabilities for the TTC subsystem.

7.A

Appendix

An example SPN model of the Gyro satellite subsystem using the SPN@ software developed by Volovoi (2004; 2006) is shown below.


February 18, 2011

7:42



119

Multi-state failure analysis of the remaining subsystems is shown in the figures below (Figures 7.A.1 to 7.A.16). Beam 1.000 0.998

0.016 Operational (S4)

0.014

Minor (S3) Major (S2)

0.996

0.012

0.992

Probability

Probability

0.994

0.990 0.988

0.010 0.008 0.006

0.986

0.004

0.984 0.982

0.002



Figure 7.A.1 Probability of being fully operational for Beam subsystem.

Figure 7.A.2 Degraded states probabilities for Beam subsystem.

CP 1.000

0.020 0.018

0.995

0.016


0.980

Probability

0.985


0.014

0.990 Probability


0.012 0.010 0.008 0.006 0.004

0.975

0.002 0.970 0


Figure 7.A.3 Reliability and probability of being fully operational for CP subsystem.


Figure 7.A.4 Degraded states probabilities for CP subsystem.



120

February 18, 2011

7:42



Mechanisms 1.000

0.010

0.008

0.990 Reliability Operational (S4) 0.985

Probability

Probability

0.995


0.006

0.004

0.980

0.002



Figure 7.A.5 Reliability and probability of being fully operational for Mechanisms subsystem.

Figure 7.A.6 Degraded states probabilities for Mechanisms subsystems.

Payload 0.12

1.00 0.98

0.10


0.08

0.92


0.90

Probability

Probability

0.96 0.94

0.06 0.04

0.88 0.02

0.86 0.84 0


Figure 7.A.7 Reliability and probability of being fully operational for Payload subsystem.

0.00 0


Figure 7.A.8 Degraded states probabilities for Payload subsystem.


February 18, 2011

7:42



121

Battery 0.014

1.000 0.995


0.012

Probability

Probability

0.985 0.980

0.008 0.006 0.004

0.975 0.970 0


0.010

0.990



Figure 7.A.9 Reliability and probability of being fully operational for Battery subsystem.

Figure 7.A.10 Degraded states probabilities for Battery subsystem.

ED 0.012

1.000 Reliability Operational (S4)

0.010

0.995


0.990

0.985

Probability

0.008 Probability


0.006 0.004

0.980

0.002



Figure 7.A.11 Reliability and probability of being fully operational for ED subsystem.

Figure 7.A.12 Degraded states probabilities for ED subsystem.



122

February 18, 2011

7:42



SAD 1.000

0.006

0.999

0.005

0.998

0.004

0.996 0.995


0.994

Probability

Probability

0.997


0.003 0.002

0.993 0.992

0.001



Figure 7.A.13 Reliability and probability of being fully operational for SAD subsystem.

Figure 7.A.14 Degraded states probabilities for SAD subsystem.

SAO 1.00

0.08

0.99

0.07

0.98

0.06


0.96 0.95


0.94 0.93

Probability

Probability

0.97 0.05 0.04 0.03 0.02

0.92 0.01

0.91 0.90 0


Figure 7.A.15 Reliability and probability of being fully operational for SAO subsystem.

0.00 0


Figure 7.A.16 Degraded states probabilities for SAO subsystem.



February 17, 2011

11:37


8

Toward survivability analysis of spacecraft and space-based networks1 8.1

Introduction

The objective in this chapter is to extend the previous analyses of spacecraft reliability and multi-state failures of spacecraft subsystems to include considerations of survivability of spacecraft and space-based networks. In addition, this chapter introduces an important tool for the modeling and analysis of stochastic processes, namely, stochastic Petri nets (SPNs), and develops SPN models for the analysis of spacecraft survivability, building on detailed models of subsystems’ multi-state failures. A framework for the quantitative analyses of system survivability is proposed, and it is put to use, in a proof-of-concept way, for the comparative analysis of the survivability of a monolith spacecraft and a space-based network. The framework proposed, as well as the modeling and simulation capability demonstrated in this chapter, should prove useful, not only to the space community, but also to government agencies and other stakeholders who have an interest in the survivability of systems and networks. Modeling, analyzing, and predicting failures is a central focus of many engineering disciplines dealing with system design and operations, such as civil, aerospace, and electrical engineering. Two related objectives from such a focus are: (1) to assess and rank different design options based on their propensity and ability to cope with failures – the “analyst’s” perspective; and (2) to make design choices that would (2a) 1 This chapter is based on an article written by the authors and published in Reliability Engineering and System Safety (Castet and Saleh, 2011).



124

February 17, 2011

11:37



80

800

70 60 Survivability

700

Survivability Resiliency

600

50

500

40

400

30

300

20

200

10

100 0 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

0

Resiliency


Year of publication

Figure 8.1 Survivability and resilience/resiliency publications per year since 1960. prevent the occurrence of these failures or reduce the system’s propensity to failures, (2b) mitigate the consequences of failures if they occur or limit their propagation throughout the system, and (2c) enable timely and effective recovery from failures – the “designer’s” perspective. Given the design and development of increasingly complex and interconnected systems, it has become even more important to analyze the propensity to failures of such systems and whether they would experience catastrophic failures or graceful degradation following node or component failures for example. These failures may be triggered by endogenous or exogenous causes (e.g., attacks), and the analysis would assess, among other things, how localized failures or disruptions would propagate throughout the system. These concerns fall within the realm of survivability and resiliency analysis. A brief overview of these two concepts is provided in the next section.

8.2

Overview of survivability and resiliency

Survivability and resiliency are used extensively in the scientific and technical literature as multi-disciplinary concepts in a variety of contexts and often with different meanings. A lexical search in the academic database ISI Web of Science illustrates the growing use of these concepts in the technical literature. Figure 8.1 summarizes the results of this literature search: the first documented use of these concepts in the database2 started in the 1960s with a handful of articles published on these subjects in the first decade, followed by a dramatic increase in the mid 1990s that has continued until today (over 60 articles were published on survivability in 2009 2 Used

in the titles of of the articles. A similar more pronounced trend is found when the search probed for these concepts in the keywords of the publications instead of the titles.



February 17, 2011

11:37


SURVIVABILITY ANALYSIS OF SPACECRAFT AND NETWORKS

125

and more than 510 on resiliency). In addition, the interest in one particular topic, namely, survivable or resilient networks, appeared in the 1980s and followed the same exponential trend. These searches conducted on ISI Web of Science also identify the academic disciplines that grapple with survivability and resiliency. The concept of survivability is traditionally associated with engineering, whereas resiliency is more often found and discussed in environmental sciences as well as in psychology and psychiatry. Note that the words resilience or resiliency are equally found and used interchangeably in publications. As a consequence, only the word resiliency will be used in this chapter.

8.2.1

On survivability

8.2.1.1

Military context

Survivability as a system attribute has always been important to the military, and its experimental and analytical assessment has probably heightened since the 1960s (Ball and Atkinson, 1995). Survivability in a military context is at present applied to platforms (e.g., aircraft), people, systems (e.g., military networks), and now more generally to missions. Several articles show this evolution, from one of the first attempts to assess survivability of an aircraft in 1967 (Ball and Atkinson, 1995; Atkinson et al., 1969) to some more general definitions (MIL-STD-2069, 1981; MIL-HDBK-2069, 1997; MIL-HDBK-336-1, 1982) such as the one provided by the DoD Regulation 5000.2-R (1999): “[survivability is] the capability of a system and crew to avoid or withstand a man-made hostile environment without sustaining an impairment of its ability to accomplish its designated mission. Survivability consists of susceptibility, vulnerability, and recoverability.” Susceptibility is “the degree to which a weapon system is open to effective attack because of one or more inherent weakness”; vulnerability is “the characteristic of a system that causes it to suffer a definite degradation (loss or reduction of capability to perform its designated mission) as a result of having being subjected to a certain (defined) level of effects in an unnatural (man-made) hostile environment”; recoverability is “the ability, following combat damage, to take emergency action to prevent the loss of the system, to reduce personnel casualties, or to regain weapon system combat mission capabilities.” In addition, several publications addressed the issue of survivability of military communication networks, a growing area of interest and research since the 1990s, and for which survivability of the network is defined as the “ability to maintain communication among the nodes when it is subject to deliberate destruction” (Kang et al., 1998). 8.2.1.2

Engineering context

Following its analysis within a military context, the concept of survivability has spread to other areas, especially to electrical engineering with an emphasis on software, telecommunications, and information systems. In particular, survivability has become of major interest for network system designers since society has become significantly dependent on a variety of networks, leading to severe consequences in the case of network system disruptions or failures. While the use of “survivability”



126

February 17, 2011

11:37



is widespread within the technical community, no definition has been unanimously adopted. Westmark (2004) compiled 53 definitions of survivability from different publications and synthesized the following definition: survivability, according to Westmark, is “the ability of a given system with a given intended usage to provide a pre-specified minimum level of service in the event of one or more pre-specified threats.” One of the more cited definitions of survivability is provided by Ellison et al. (1999): survivability, according to Ellison et al., is the “capability of a system to fulfill its mission, in a timely manner, in the presence of attacks, failures, or accidents.” Knight et al. (2003), while focusing on survivability in a telecommunications and network context, found previous definitions not precise enough, and proposed a formal definition of survivability based on six quantitative parameters (or sextuple). These authors characterized a system as “survivable if it complies with its survivability specification,” and the survivability specification is mathematically defined, gathering all acceptable levels of service from the system, the associated service values and relative values (perceived by the user), its probabilistic requirements, and its possible transitions in a specified operating environment. Accordingly, survivability definitions teeter between the informal and the formal, and occasionally include probabilistic terms. But, as the previous definitions indicate, survivability is context specific, related to the system studied and its environment, the services it provides to users, and the requirements that have been set for it. This specificity explains why survivability seems often to be a more generic word defined or measured in terms of other notions, like availability, performance, security, reliability, traffic capacity, connectivity, etc. Survivability is also related to or defined with respect to other concepts, for example, the notions of recoverability and resiliency. These are briefly discussed in the next subsections to clearly delineate survivability and its usage in this work from these other concepts.

8.2.2

On resiliency

Resiliency is also a concept with multiple definitions. Caralli (2006) notes that resiliency was first used to describe a physical property of a material. It is “the capability of a strained body to recover its size and shape after deformation caused especially by compressive stress” according to the Merriam-Webster Dictionary. Resiliency has expanded beyond this physical definition to other disciplines, particularly in ecology (Walker et al., 2004) and psychology, where “resiliency refers to the ability of people to bounce back from adversity” (Caralli, 2006). Resiliency is now used to describe engineering systems, networks, and organizations with a similar meaning to bouncing back to normal operation following disruption. Several aspects are present across the diversity of definitions of resiliency, for example, the ability to “change when a force is enacted, [to] perform adequately or minimally while the force is in effect, [to] return to a predefined expected normal state whenever the force relents or is rendered ineffective” (Caralli, 2006). Thus time becomes an important parameter in the assessment of system resiliency, which accounts for the dynamics of the system response, from the disruption or shock to the after-shock.



February 17, 2011

11:37



127

Shock Performance

ΔP (survivability)

τ (recoverability) Time

Figure 8.2 Notional system response following a shock.

8.2.3

Comparing survivability and resiliency

Survivability and resiliency are often compared in the technical literature, and the most frequent result is that resiliency is described as a broader concept, a superset including survivability. For example, according to the ResiliNets Initiative (Xie et al., 2005), resiliency is the “ability of the network to provide and maintain an acceptable level of service in the face of various challenges to normal operations,” adding that resiliency is “survivability plus the ability to tolerate unusual but legitimate traffic load.” Caralli (2006) notes that resiliency is an extension of survivability, which deals with the system response to a shock or disruption, to include “restoration of normal processes once a disruption has relented.” Time dependency of a system response to disruptions is therefore intrinsic to the definition of resiliency. As noted previously, survivability is defined for systems in given environments, with respect to particular threats, and given a performance index chosen by the user to assess the performance of the system considered. Since this definition is context specific, the environment, the threat(s), and the performance index have to be specified each time an analysis is conducted. Figure 8.2 provides a notional representation of a system response facing a shock or disruption. The survivability of the system is related to the performance degradation3 P. The response of the system after the shock characterizes the recoverability of the system, which in simple terms can be thought of as the time needed for the system to return to within a certain percentage of its initial level of performance. Resiliency as noted previously is commonly defined as the superset combining survivability and recoverability. Only survivability is considered in this chapter. 3 As an aside, graceful degradation, which is particularly desirable for systems with high-availability requirements, allows a system to keep operating and providing some level of service by staging the system’s performance degradation over time.



February 17, 2011

11:37


128


8.3

Survivability framework

Survivability, as discussed previously, is assessed for a system in a given environment, with respect to particular classes of threats or disruption, and given a metric or metrics to reflect the performance degradation of the system. Therefore, as noted previously, the environment, the threats or disruptions, and the performance metric(s) have to be specified each time a survivability analysis is conducted. A high-level framework for survivability analysis is shown in Figure 8.3, and it captures the different steps through which survivability analysis proceeds. Figure 8.3 starts on the left with the definition or delineation of the classes of threats or types of disruption that the analyst is interested in assessing with respect to the system’s survivability. Survivability, like the concept of optimization, remains ill-defined unless additional information is provided: what the system is optimized to, with respect to the latter; and what the system is survivable for, with respect to the former. The characterization of the classes of threats or types of disruption of interest constitutes the first step in a survivability analysis. The second step in Figure 8.3 is design-centric and seeks to characterize the architecture of the system under consideration, its (functional) structure, and design choices (e.g., modularity, coupling, redundancy, etc.), in particular its features that pertain to performance. The third step in Figure 8.3 transforms the previous step into an analytical or computational model of the system to assess its survivability with respect to the classes of threats or types of disruption of interest. Westmark (2004) provides an extensive review of the different mathematical tools used in the literature for this third step. These tools include state machines, trellis graphs, Markov processes (with Monte Carlo simulations). SPNs are used in this chapter for this third step for reasons discussed in the next section. Finally, the last step in Figure 8.3 consists of assessing the system’s performance degradation – its survivability assessment – following disruption, using the system model previously developed and the characterization of the classes of threats or types of disruption of interest (step 1). To illustrate one use of this framework, this chapter confines the classes of shocks or disruption considered to on-orbit anomalies and failures in spacecraft subsystems (building on the results in previous chapters: subsystem models based on the transition diagram presented in Figure 7.2 and Weibull models for transitions given in Table 7.1 and Table 7.2), and it conduct a survivability analysis for two types of space architecture: a traditional monolith spacecraft and a space-based network, details of which follow shortly. Since the spacecraft models were developed using SPNs, a brief discussion of this modeling and simulation environment is provided next.

Classes of threats / shocks (definition and characterization)

System / network architectural and design choices

System / network model development (for performance analysis)

Figure 8.3 Survivability framework.

System response: performance degradation following shocks



February 17, 2011

11:37



8.4

129

Introduction to stochastic Petri nets (SPNs)

Petri nets were introduced in 1962 by the German computer scientist, Carl Adam Petri (1926–2010). A Petri net is a bipartite directed graph used to model discrete-event systems, which can display concurrent or asynchronous processes (Peterson, 1977). The Petri net graph has two disjoint set of vertices (or nodes) termed places and transitions. Directed arcs are drawn between a place and a transition (input arc), or between a transition and a place (output arc). Places connected to a transition by input arcs are called input places of that transition, and places connected to a transition by output arcs are called output places of that transition. Places in a Petri net model particular conditions or states that the system can assume, while transitions model events that affect the system. In addition to places, transitions, and directed arcs, Petri nets also have “tokens,” or markings, that can be associated with each place; the condition associated with a place is realized, and the system assumes the particular state reflected by that place, when one or more token(s) are in that place, as shown in Figure 8.4. A Petri net has static and dynamic properties: its graph describes its static properties, and its execution reflects its dynamic properties (Peterson, 1977). The execution of the Petri net is marked by the movement of the tokens from place to place, through the “firing” of transitions. The firing of transitions occurs only if the transition has been “enabled” through the presence of tokens in all its input places. The set of locations of tokens in a Petri net is termed its marking, and it characterizes the overall state of the system at a given time. Formal mathematical definitions of Petri nets can be found in Peterson (1977; 1981) or Haas (2002). Stochastic Petri nets (SPNs) are a subfamily of Petri nets, and they add a stochastic behavior by introducing randomness in the firing of transitions, modeled for example with exponential, Weibull, or lognormal distributions. Although initially used for the modeling and analysis of manufacturing systems and computer networks, SPNs are slowly but increasingly being adopted for reliability studies and risk analysis because of the many advantages they provide over Markov chains, for example, as will be discussed shortly (Volovoi, 2004). Details on SPNs can be found in Haas (2002) and Ajmone Marsan (1989). Figure 8.4 shows an illustrative example of a SPN. token

transition 1

Place 1

transition 2

transition 3

Place 2

transition 4

Place 3

Figure 8.4 Example of a SPN.



130

February 17, 2011

11:37



Two types of arcs are of particular interest to system modelers, the inhibitor arc and enabler arc. An inhibitor arc prevents a transition from firing when a token is present in the place linking the transition and the place. An enabler arc, as discussed in Volovoi (2006), is a “negative inhibitor” that enables or forces the transition to occur. In essence, the inhibitor and enabler arcs “override” the stochastic nature of a transition in a SPN model (or a deterministic temporal delay in a regular Petri net). Examples of an inhibitor and an enabler arc are presented in Figure 8.5. The upper transition has a deterministic time delay τ = 1 s and the lower transition a time delay τ = 2 s. In Figure 8.5a, the execution of the SPN including an inhibitor arc is shown at three points in time (t = 0, 2, and 3 s). Notice that despite the fact that the upper transition is faster (τ = 1 s) than the lower one (τ = 2 s), the presence of a token activating the inhibitor at the initial time prevents the upper transition from firing after 1 s. The lower transition fires after 2 s, deactivating the inhibitor and enabling the upper transition to fire 1 s later (after the inhibitor is deactivated). Figure 8.5b models the same system behavior with an enabler: the upper transition is activated by the enabler after the lower transition has fired (only when the enabler arc is active does the upper transition “start its clock”).

τ=1s

τ=1s

inhibitor

t=0s

t=0s

-1

τ=2s

τ=2s

t=2s

t=2s

-1

t=3s

t=3s

-1

(a)

(b)

Figure 8.5 Example of an inhibitor arc (a) and enabler arc (b).

enabler



February 17, 2011

11:37



8.5

131

SPNs for spacecraft modeling and survivability analysis

SPNs are chosen in this work as the modeling tool over the more traditional Markov chain (MC) primarily because SPNs can handle more easily local modeling, time dependencies, and local clocks for the state evolution than MC. In other words, modeling features that are required for the survivability analysis are more easily implemented in SPN than MC. For example, local analysis is important in this study to model the interaction between the spacecraft subsystem and system levels (see Figure 8.7 and Figure 8.8 for example); and time dependency is needed to model the evolution of spacecraft subsystem failure rates over time (no constant failure rates or nonhomogeneous probabilities of transitions for spacecraft subsystem multi-state failure as seen in Chapter 7). Also, because of the number of subsystems considered here, the state space for a MC (or a semi-Markov model) would make it unmanageable and impossible to visualize. As an example, to represent the 12 subsystems, including the Unknown category and the four different states for each subsystem, 48 places are necessary in the case of a SPN to capture the overall state of the spacecraft, whereas more than 16 million states are necessary in a Markovian approach. This state explosion, when there is a need to model subsystemand system-level performance with multiple levels of anomalies and failures, is likely to lead in the future in the displacement of MC modeling by SPNs for survivability analysis. Because of the stochastic nature of SPNs, Monte Carlo simulations are necessary to generate a representative behavior of the stochastic transitions and other statistics of interest (e.g., residency time in each state). In this chapter for example, due to the complexity of the SPN models described hereafter (see Figure 8.8 for the SPN model of the monolith spacecraft), 5 million runs were conducted to obtain an acceptable level of precision on the statistics of interest. The impact of the number of runs in the Monte Carlo simulations will be addressed later in this section. As part of this modeling, each spacecraft subsystem was modeled with a SPN based on the transition diagram presented in Figure 7.2. An example SPN model of the Gyro satellite subsystem was provided in the appendix to the previous chapter. A list of all the spacecraft subsystems considered in this work can be found in Chapter 5, and the different states for each subsystem are discussed in Chapter 7. The two example architectures of space systems studied in this chapter are shown in Figure 8.6. The monolith architecture consists of a single spacecraft with the 11 subsystems mentioned previously, plus the Unknown category. The first architecture consists of a monolith spacecraft. The second architecture is related to a novel concept recently introduced in the space industry termed fractionation, or co-located spacebased network (Brown and Eremenko, 2006a; 2006b). By physically distributing functions in multiple orbiting modules wirelessly connected to each other, this new architecture allows the sharing of resources on orbit, such as data processing, data storage, and downlinks. Preliminary analysis suggests that such an architecture,


February 17, 2011

11:37


Telemetry Tracking and Command #1

Telemetry Tracking and Command #2

Unknown

Unknown

Solar Array Operating


Solar Array deployment


Telemetry Tracking and Command


EPS

EPS


Electrical distribution


Battery / Cell


MONOLITH



Control Processor

Control Processor

Beam / Antenna operation / deployment


Thruster / Fuel

Thruster / Fuel

AOCS

Gyro / Sensor / Reaction wheel

Battery / Cell

BOX 2


Control Processor



Battery / Cell

BOX 1


EPS

Unknown


SBN S/C #1

Thruster / Fuel


SBN S/C #2

Figure 8.6 Architecture of the monolith spacecraft (top) and the space-based network (bottom).


AOCS

132

AOCS




February 17, 2011

11:37



133

under certain conditions and despite some initial overheads, offers several advantages over the traditional monolith spacecraft design (details can be found in Dubos and Saleh, 2011). The purpose of developing a SPN model for a space-based network, in addition to that of a monolith spacecraft, is to illustrate or provide a proof of concept of a comparative survivability analysis of these two architectures. The space-based network (SBN) considered here is simple and consists of two networked spacecraft:

r The first spacecraft (SBN S/C #1 in Figure 8.6) contains all the subsystems, as the monolith spacecraft. However, it has the ability to communicate with another spacecraft (S/C #2 in Figure 8.6). A particular truncation of S/C #1 is termed BOX 1, and it consists of all the subsystems except the TTC subsystem, called TTC1, as shown in Figure 8.6.

r The second spacecraft has the necessary subsystems for a spacecraft (called here BOX 2: attitude control, control processor, power, antenna, and mechanisms/structures/thermal) plus a TTC subsystem, called TTC2, which S/C #1 can tap into or use in case of damage to or failure of TTC1. As seen in previous chapters, the TTC subsystem is the major driver of spacecraft unreliability. The wirelessly connectivity in the SBN enables a type of redundancy in the TTC between the two spacecraft in the network. To evaluate the survivability of these two architectures, four states were considered by the authors at the system level:

r System state 4, operational: r System state 3, minor degradation: r System state 2, major degradation: r System state 1, failed:

0–5% performance loss 5–35% performance loss 35–85% performance loss 85–100% performance loss.

These states determine the level of precision for the survivability analysis of the models. Additional precision can be obtained by defining additional states, which comes at the cost of increased analytical and computational complexity, as discussed previously. The probabilities of being in these four states are the output of the SPN model. Comparisons between the probabilities obtained provide the comparative survivability analysis of these two architectures, as will be shown shortly. In the case of the monolith spacecraft, the following rules are used to link the subsystem and system levels of degradations and failures:

r The system is in the operational state if all the subsystems are in their operational states.

r The system is in the failed state if one subsystem is in the Class I state.



134

February 17, 2011

11:37



Table 8.1 Impact on the system level of subsystem major degradation (conditional probabilities). Conditional probability that a Class II subsystem leads to system: Subsystem Gyro Thruster Beam CP Mechanisms Payload Battery ED SAD SAO TTC Unknown

minor degradation

major degradation

failed

25.7% 50.9% 70.6% 0% 100% 33.4% 56.2% 40% 40% 61% 43.5% 58.4%

54.3% 47.3% 23.5% 0% 0% 59.1% 18.8% 40% 60% 31.2% 34.8% 33.3%

20% 1.8% 5.9% 100% 0% 7.5% 25% 20% 0% 7.8% 21.7% 8.3%

r The Class IV–Class III state of the subsystems does not have a direct effect on the system level.

r The Class II state can lead to minor or major degradation or failed system states according to conditional probabilities peculiar to each subsystem, as given in Table 8.1 (actual on-orbit data derived from the database). The probabilities given in Table 8.1 are “conditional” since they represent the probability that the system will transition to a degraded state given that a particular subsystem is in the Class II state. For example, for the Gyro subsystem, given that this subsystem is in Class II (major anomaly), there is a 25.7% chance that the system transitions to a minor degradation state, a 54.3% chance to a major degradation state, and a 20% chance to a failed state. A summary of the subsystem and system states, and the links between them, is provided in Table 8.2. Given the stochastic transition laws between the different states summarized in Table 8.2, the SPN model of a monolith spacecraft facing on-orbit failures and anomalies was developed and is shown in Figure 8.7. To clarify this model and enable easy identification of its different parts, Figure 8.8 shows the overall SPN model, the spacecraft architecture, the various subsystem models, and the system transition diagram. In addition, a zoomed-in enlargement of two subsystem SPN models is shown in Figure 8.8, namely, those of the Gyro and the TTC subsystems. The system-level states are clearly identified and illustrated by a schematic transition diagram. The remaining states are labeled “intermediary states” and are used to

All defined by the authors

r System state 1: failed (system failure)

r System state 4: operational r System state 3: minor degradation r System state 2: major degradation

→ → →

subsystem state 3

subsystem state 2 subsystem state 1

If a subsystem transitions to . . .


system state 1

⎫ system state 3 ⎬ or system state 2 (see Table 8.1) ⎭ or system state 1

no transition (no impact on system states)

then the system transitions to . . .

Transitions between system states depend on subsystems states:

11:37

Four states based on the performance degradation of the system:

Weibull distributions derived from statistical data analysis (see Chapter 7)

February 17, 2011

All derived from the classes of events present in the database (see Chapter 7, Section 7.2)

r Subsystem state 4: operational r Subsystem state 3: minor anomaly r Subsystem state 2: major anomaly r Subsystem state 1: total failure

For each subsystem, four states based on subsystem anomalies and failures:

Transitions between states


System level

Subsystem level

States

Table 8.2 Summary of subsystem and system states and transitions.


135


136

February 17, 2011 11:37

Figure 8.7 SPN model of a monolith spacecraft: multi-state failure model linking subsystem anomalies and failures to system performance degradation.





February 17, 2011

11:37



137

Figure 8.8 Construction clarification of the SPN model of a monolith spacecraft. link the subsystem level to the system level according to the empirical data and the previously stated rules. The results of running this SPN model are shown in Figure 8.12 below and discussed in Section 8.5.3. However, before discussing the results, the rules governing the system failure of the SBN are provided first, followed by a brief note regarding the validation of both SPN models and the number of runs used for the Monte Carlo simulation.



138

February 17, 2011

11:37



For the SBN shown in Figure 8.6, additional rules are used to link subsystem- to system-level anomalies and failures, and are presented next:

r System Failed: — If BOX1 Failed — If TTC1 Failed & TTC2 Failed or BOX2 Failed

r System Major degradation: — If BOX1 Major degradation — If TTC1 Major degradation & TTC2 Failed or TTC2 Major degradation or BOX2 Failed — If TTC1 Failed & TTC2 Major degradation & BOX2 not Failed

r System Minor degradation: — If BOX1 Minor degradation — If TTC1 Minor degradation & TTC2 Minor degradation or TTC2 Major degradation or TTC2 Failed or BOX2 Failed — If TTC1 Major degradation & TTC2 Minor degradation & BOX2 not Failed — If TTC1 Failed & TTC2 Minor degradation & BOX2 not Failed These rules can be summarized by using block diagrams, as shown in Figure 8.9. Based on these new rules, and the specific design choices of subsystems in each spacecraft (as shown in the lower part of Figure 8.6), the SPN model was developed for this particular SBN. The model is relatively complex and is provided in the appendix to this chapter instead of the main body for readability purposes. In general terms, the SPN model of the SBN is built in a similar fashion and mirrors the construction of the SPN model for the monolith spacecraft.

8.5.1

Testing the SPN models

Given the relative complexities of both the SPN models, several tests were devised to verify that the models were properly built, and that they reflected actual on-orbit data when available, as will be discussed shortly. In essence, these tests provided partial verification and validation of the SPN models (IEEE Std 1012-2004). The monolith SPN model has been tested extensively and later used to validate the SPN model of the SBN, particularly with respect to the state probabilities of BOX1, BOX2, TTC1, and TTC2.


February 17, 2011

11:37



SBN F

⇔

139

Legend:

TTC1 F

F – failed

BOX1 F TTC2 F

M – major degradation

BOX2 F

m – minor degradation F – not failed

TTC1 F TTC1 M SBN M

⇔

TTC2 M

BOX1 M TTC2 M

TTC2 F

BOX2 F BOX2 F

TTC1 M

TTC1 F

TTC2 m

TTC2 m

BOX2 F

BOX2 F

TTC1 m SBN m

⇔

BOX1 m TTC2 m

TTC2 M

TTC2 F

BOX2 F

Figure 8.9 Block diagrams for the system-level state determination of the SBN (shown in Figure 8.6). One test of the monolith SPN model was done by evaluating the monolith reliability obtained in the Monte Carlo simulation, and comparing it to the Kaplan–Meier spacecraft reliability derived in Chapter 2. Figure 8.10 presents the two results: (1) the simulated spacecraft reliability as derived from the SPN model in Figure 8.7; and (2) the actual benchmark nonparametric spacecraft reliability. As can be seen in Figure 8.10, the SPN model output closely approximates – staying within 1 percentage point – the actual spacecraft reliability (or more precisely, its Kaplan–Meier 1.00 0.99 0.98

Nonparametric reliability SPN simulated reliability

0.97 Reliability


0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0


Figure 8.10 Nonparametric spacecraft reliability and (one of the) SPN model outputs.



140

February 17, 2011

11:37



estimate). This result indicates that the SPN model and its state transitions have been properly coded, and one of its outcomes, spacecraft reliability, accurately represents statistical data of observed on-orbit failures. For the SPN model of the SBN, the state probabilities of BOX1, BOX2, TTC1, and TTC2 were tested against the results from the truncated monolith SPN model. The results from these four items were significantly accurate and remained within 0.1 percentage point of the “benchmark” results, and demonstrated that this model accurately replicated the benchmark results of the monolith spacecraft. As an example of further testing of the SPN model for this SBN, the simulated system state probabilities were compared to the theoretical probabilities obtained based on the block diagrams presented in Figure 8.9. For example, the probability of failure of the system can be calculated as follows: PSBN failed = 1 − (1 − PBOX1 F ) · {1 − [1 − (1 − PTTC2 F ) · (1 − PBOX2 F )] · PTTC1 F } . (8.1) The probabilities on the right-hand side of Equation (8.1) can be monitored in the simulation of the SPN model. These simulated probabilities of the two boxes and TTCs can then be used to calculate, according to (8.1), a theoretical probability of failure of the SBN, the right-hand side of (8.1). This theoretical result from the right-hand side of (8.1) is then compared to the outcome of the SPN model shown in the appendix of the simulated probability of the system being in the failed state (simulated left-hand side of (8.1). The theoretical and simulated results were within 0.01 percentage point of each other (comparable results can be derived for the other system states and block diagrams shown in Figure 8.9), thus indicating that the rules linking the subsystem-level degradation and failures to the system level were properly coded in the SPN model of the SBN.

8.5.2

Monte Carlo simulation runs for the SPN models

As noted previously, because of the stochastic nature of SPN, Monte Carlo simulations are necessary to generate a representative behavior of the stochastic transitions and other statistics of interest. Given the relative complexity of the SPN models, four Monte Carlo simulations, with 100 000 runs, 1 million runs, 5 million runs, and 10 million runs, were conducted to assess the accuracy of the statistics of interest as outputs of the models, namely, the probability of residency in each system state. The Monte Carlo simulation with 10 million runs for the SBN took on average 9 minutes (on an Intel Core 2 Duo 2.66 GHz processor). In the following, the results of the simulation with 10 million runs serve as a benchmark for the other three simulations. Figure 8.11 presents the difference in the probabilities of the system-level states between the simulation with 10 million runs and each of the other simulations with smaller numbers of runs. It can be observed that all the results stay within 0.3 percentage point from the result with 10 million runs. In other words, increasing the number of runs by


February 17, 2011

11:37


SURVIVABILITY ANALYSIS OF SPACECRAFT AND NETWORKS Operational state

Minor degradation state

0.20 0.15 0.10 0.05 0.00 -0.05 -0.10 -0.15 -0.20 -0.25

0.25 Difference (percentage point)

100,000 - 10M 1M - 10M 5M - 10M


141

0.30

0.30

0.20 0.15 0.10 0.05 0.00 -0.05 -0.10 -0.15 100,000 - 10M 1M - 10M 5M - 10M

-0.20 -0.25

-0.30 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)


Major degradation state

Failed state 0.30

0.30 0.20 0.15 0.10 0.05 0.00 -0.05 -0.10 -0.15 -0.20


100,000 - 10M 1M - 10M 5M - 10M



0.20 0.15 0.10 0.05 0.00 -0.05 -0.10 -0.15 -0.20

100,000 - 10M 1M - 10M 5M - 10M

-0.25

-0.25



Figure 8.11 Difference between the SPN model outputs (system state probabilities) for different Monte Carlo simulations (100 000 runs, 1 million runs, 5 millions runs) with respect to 10 million runs. two orders of magnitude from 100 000 to 10 million provides an exceedingly small improvement in terms of precision of the estimated probabilities. Within this small range, a noticeable improvement occurs when the number of runs is increased from 100 000 to 5 million, and as such the 5 million runs were chosen for the simulations as a suitable compromise between accuracy and simulation running time (100 000 runs in 5 seconds, 1 million runs in about 1 minute, 5 million runs in about 4 minutes, and 10 million runs in about 9 minutes).

8.5.3

Results

Running the Monte Carlo simulation of the SPN model in the case of the monolith spacecraft provides the evolution in time of the probabilities of the system being in operational or different failed states (i.e., operational, minor and major degradation, failed). Figure 8.12 presents these results, shown in two different plots for readability purposes given their different ranges on the y-axis.



142

February 17, 2011

11:37


SPACECRAFT RELIABILITY AND MULTI-STATE FAILURES Operational state

Degraded states

1.00

0.14

0.95

0.12 0.10

0.85

Probability

Probability

0.90

0.80 0.75

0.06 0.04

0.70

minor degradation major degradation failed

0.02

0.65 0.60 0

0.08

0.00 0



Figure 8.12 State probability results of the monolith spacecraft SPN model.

Figure 8.12 reads as follows. After 6 years on orbit, for example, a monolith spacecraft has a 75.6% likelihood of being fully operational, 8.4% of being in a minor degradation state, 8.1% of being in a major degradation state, and 7.9% of being in a failed state. Similarly, after 10 years, for example, a spacecraft has only a 70% likelihood of being fully operational, that is, of not experiencing some form of anomaly or degradation. This result offers a significant opportunity, and can be thought of as a call to arms, to improve spacecraft design and testing. For the SBN, running the simulation of its SPN model leads to the same kinds of plots. This in turn allows a comparison of the probability of residency in each state for both architectures. Figure 8.13 for example displays the probability of residency in the operational and failed states for the monolith spacecraft and the SBN.

Operational state

Failed state

1.00

0.14 Monolith Space-based network

0.95

0.12 0.10

0.85

Probability

Probability

0.90

0.80 0.75

0.06 0.04

0.70

0.02

0.65 0.60 0

0.08


(a)

0.00 0

Monolith Space-based network 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

(b)

Figure 8.13 Comparison between the monolith spacecraft and the SBN for the operational (a) and failed (b) states.


February 17, 2011

11:37



143

Figure 8.13a is confined to the operational state and clearly shows that the SBN is more likely to be in an operational state than the traditional monolith spacecraft at any point in time, given stochastic on-orbit anomalies and failures. For example, after 15 years, there is a 65.9% likelihood that the SBN will still be in an operational state, compared to 63.9% for the monolith spacecraft. This 2 percentage point increment is provided by the networked nature of this architecture (bottom part of Figure 8.7) and the ability of one spacecraft to tap into a resource, in this case the TTC, of the second spacecraft. Similarly, Figure 8.13b shows that the SBN is less likely to be in a failed state than the traditional monolith at any point in time. For example, after 15 years, there is an 11.2% likelihood that the SBN will be in a failed state, compared to 13.1% for the monolith spacecraft. A more visually compact and exhaustive analysis of the simulation results for the two architectures is provided in Figure 8.14. The figure represents the difference in the resulting probabilities of being in any one of the four states between the two architectures. The y-axis is the difference in percentage point between the two architectures. The important results that can be gleaned from Figure 8.14 are as follows:

r The probability of being in an operational state is higher at any point in time for the SBN than for the monolith spacecraft (the difference between the probabilities of the operational states of the network and the monolith is positive). In other words, at any point in time, the SBN is more likely to be operational than the monolith spacecraft. As noted earlier, after 15 years, the incremental likelihood of the network over the monolith being in an operational state is

Difference between monolith and SBN state probabilities (percentage point)


3 2 1 0 -1 -2 -3 0

Operational Minor degradation Major degradation Failed


Figure 8.14 Survivability superiority of the co-located SBN over the monolith spacecraft.



144

February 17, 2011

11:37



2 percentage point. A careful cost–benefit analysis should be conducted to assess whether this incremental probability of remaining in the operational state is worth the cost of obtaining it. While such studies are beyond the scope of this work, it is worth pointing out in this regard that communication satellites for example can generate in excess of $50 million per year and this incremental 2 percentage point can represent the equivalent of several months’ worth of revenues. Similarly, an incremental 2 percentage point for remaining in the operational state can be of significant importance for defense or intelligence space assets. More importantly, this incremental probability of remaining operational was obtained by considering the “functional redundancy” of only one subsystem, the TTC, and that significant potential improvement could be gained by considering additional subsystems and more than just two spacecraft able to tap into each other’s resources.

r For the failed state, the difference in probabilities between the SBN and the monolith spacecraft is negative, meaning that the probability of being in this state is lower for the SBN than for the monolith spacecraft. The same interpretation as in the previous bullet point applies in these cases as well. The difference for the minor and major degradation states is smaller than for the other states. It is important to interpret these results cautiously and not expand their domain of applicability or overestimate their generalities. More specifically, the results provided here show that the particular design of a SBN shown in Figure 8.6 is more survivable than that for a monolith spacecraft with respect to anomalies and failures actually observed in the spacecraft subsystems (i.e., actual on-orbit data). As discussed in Section 8.3, this constitutes a limited scope of survivability analysis, which is confined to a particular class of “shocks” of on-orbit disruption. These results should not be generalized to all designs of SBNs or monolith architectures and they should not be extrapolated to other classes of on-orbit shocks or threats to space systems. Nevertheless, it is believed that the survivability framework proposed here (Figure 8.3) and the stochastic modeling tool adopted offer fruitful avenues for further research and possible adaptation toward the survivability analysis of a broad range of architectural and design choices for space systems, and other engineering artifacts, and given different classes of shocks.

8.5.4

Limitations

The modeling and analysis of spacecraft failure behavior developed in this chapter have several limitations; these are noted here for the reader’s benefit and should be kept in mind as opportunities for future improvements:

r Failure independence: it was implicitly assumed that subsystem failures are independent. In reality, some subsystems may have dependent anomalies and failures, for example, the thermal and power subsystems. Unfortunately, the



February 17, 2011

11:37



145

information available in the database, and sometimes in the satellite operator’s incident report itself, does not explicitly address failure dependence. For example, a spacecraft Class I failure is ascribed to only one subsystem, and a partial failure of a subsystem has its timing and severity recorded. As noted previously, the statistical analysis in this work is enabled by and confined to the data available. As a result, common-cause and dependent anomalies and failures of spacecraft subsystems cannot be clearly identified and statistically analyzed. Such analyses, however, are important and constitute fruitful avenues for future research when the requisite data is collected (dependent failures, however, can be easily modeled within the SPN framework).

r Wireless link between the two spacecraft: one limitation of the present analysis is in the assumption that the link between the two networked spacecraft is perfectly reliable. In reality, this may not be the case, and as a result the survivability advantages of the SBN over the monolith spacecraft may not be fully realizable. Since no reliability data for such a subsystem is available, if it exists, the choice was made in this work not to mix Bayesian probabilities for this hypothetical wireless connectivity subsystem between two spacecraft, with the data-driven (frequentist) probabilities of anomalies and failures of the other subsystems. As a consequence, the survivability advantages of the SBN over the monolith spacecraft derived here should be considered upper bound results. The SPN models, however, can easily be amended to accommodate a multi-state failure model for the wireless link subsystem.

r Uncertainty analysis: as seen in previous chapters, the statistical multi-state failure analysis is conducted on a censored data set, and thus the estimates of the reliability and probabilities of transitioning between different states of the spacecraft subsystems are associated with various levels of uncertainty. The survivability results obtained in this chapter were confined to the best estimates of these probabilities. Propagating their uncertainties to the final results would bring an additional piece of information into the survivability comparison of the different space systems considered (unfortunately, at present and to the best of the authors’ knowledge, no SPN software can run with nonparametric data).

r Rules impact: the rules presented in Figure 8.9 determine the impact of the subsystem states on the SBN states, and thus have an important impact on the survivability comparison between the studied architectures. In this work, it was assumed that the spacecraft can only tap into each other’s TTC, and thus the SPN model and the rules are confined to this “design” choice. If the spacecraft can tap into other resources as well, for example the CP, then a new SPN model has to be developed and additional rules adopted. Incidentally, current technology may not be mature enough to allow useful power beaming between different spacecraft (Lafleur and Saleh, 2009), but should this option become viable, it will support a functional redundancy of the power subsystem within a SBN.



February 17, 2011

11:37


146


A.8

Appendix: SPN model of the space-based network (SBN) in Figure 8.6 and its schematic explanation



February 17, 2011

11:37



147

P1: TIX/XYZ JWST062-Epi


February 17, 2011

9:38


Epilogue In closing the narration of his adventures, Huckleberry Finn exclaimed (in eye dialect): so there ain’t nothing more to write about, and I am rotten glad of it, because if I’d a knowed what a trouble it was to make a book I wouldn’t a tackled it, and ain’t a-going to no more. Twain’s comment provided us with a few good laughs, and at times we may have related to the second part of the sentence – but not the first! The opposite is a more accurate reflection of the situation. Extensive work remains to be done and written about, as this book covered only a small part of the broad topic of spacecraft reliability. For example, the contributions of software to spacecraft anomalies and failures were overlooked in this work; these need to be carefully analyzed, understood, and mitigated. In addition, the forensic aspect of the identification of culprit subsystems leading to spacecraft failures was not discussed in this work. For example, Chapter 5 showed that 4% to 8% of on-orbit failures are ascribed to an “unknown” cause and subsystem. This is indicative of the extent of spacecraft state of health (SOH) monitoring and telemetry points. Spacecraft health monitoring and diagnostic issues deserve to be carefully analyzed and discussed in future work. As an aside, and related to this theme, spacecraft health monitoring has expanded over the years from a few hundred to several thousand telemetry points. This proliferation of spacecraft health data creates new human–machine interface challenges. It also creates opportunities for cognitive engineering, which has significantly contributed to the design of aircraft cockpit and warning systems in general, to address these challenges. This area does not seem to have benefited from much human factor considerations, and it is ripe for contributions facilitating the job of the operators flying the spacecraft. Finally, the statistical approach adopted in this work pushed the limit in the development of actionable results of spacecraft reliability and subsystem multi-state failures, but this actuarial approach to these issues may have reached a point of Spacecraft Reliability and Multi-State Failures: A Statistical Approach, First Edition. Joseph Homer Saleh and Jean-François Castet. © 2011 John Wiley & Sons, Ltd. Published 2011 by John Wiley & Sons, Ltd. ISBN: 978-0-470-68791-8

P1: TIX/XYZ JWST062-Epi


150

February 17, 2011

9:38


EPILOGUE

diminishing returns. The next step ought to focus on and investigate the physics of failure of spacecraft and spacecraft subsystems – their actual failure modes and mechanisms. Beyond these content-centric topics, which would benefit from dedicated treatment and publications, it is useful to highlight the importance of having in place a robust process for feeding back the field (orbit) reliability results to the testing phase and further upstream in the development phases of spacecraft. It should not be forgotten that one central end-objective of analyzing spacecraft anomalies and failures is to devise various ways to prevent them from (re)occurring in the future. We conclude this work with a suggestion that the space industry develops, through some joint public–private venture, an orbiting reliability test bed, which can serve multiple stakeholders by accommodating various equipment and technologies, either unproven or of varying maturity. The infrastructure of the test bed and its housekeeping functions can be designed in a robust way that accommodates the purpose of the system, namely, the monitoring of anomalies and failures of its plug-in modules. An orbiting reliability test bed may support a faster pace of innovation and technology adoption in the space industry. Its development is likely to raise interesting technological and organizational challenges (in cost, risk, and data sharing for example), but its long-term benefits would be significant and substantial across the space industry. We hope that by the time this work is revisited and updated in a decade or so from its publication, Chapter 8 with its space-based networks will be the basis of new work, and the idea of a monolith spacecraft will by then be as obsolete as the idea of a computer not connected to the Internet is today. We end this book with the final words of Edward Gibbon (1737–1794) in The Decline and Fall of the Roman Empire, a work that exercised many years of his life: however inadequate to my own wishes, I finally deliver to the curiosity and candour of the Public.

P1: TIX/XYZ P2: ABC JWST062-App01 JWST062-Saleh

February 17, 2011

9:19


Appendix A

Geosynchronous communication satellites: system reliability and subsystem anomalies and failures1 This appendix focuses solely on communication satellites in geosynchronous orbits (hereafter referred to as GEO). This sector and type of spacecraft, GEO communication satellites, represent a major segment of the space industry, in terms of volume contracted and launches as well as revenue generated. As such, a dedicated appendix for the reliability analysis of such spacecraft is provided here. This appendix is organized into two parts. In Part I, nonparametric analysis and parametric reliability models of geosynchronous communication satellites are derived. In Part II, the focus shifts to these satellites’ subsystems, their anomalies and failures are analyzed, and a health scorecard is developed summarizing for each subsystem its track record of on-orbit failure events.

A.1

Part I: System reliability analysis

In this part, a population of 449 geosynchronous communication satellites successfully launched between January 1990 and October 2008 is analyzed. 1 This appendix was written in part in collaboration with Rachel Haga. It is based partly on an article presented at the IEEE 2011 Aerospace Conference (Haga and Saleh, 2011), and on an article presented at the International Communications Satellite Systems Conference 2009 (Castet and Saleh, 2009c).



152

February 17, 2011

9:19


APPENDIX A

Table A.1 Failure data (in days). 1 722 2472 3268

123 1122 2577 3455

128 1797 2580 3684

364 2181 2917 4192

701 2429 2947 4324

Nonparametric reliability analysis is conducted first and is then followed by parametric model fitting.

A.1.1

Nonparametric analysis of satellite failure data

The Kaplan–Meier estimator is used to treat the censored data. The reader is referred to Chapter 2 for more details on censoring and the Kaplan–Meier estimator. For the 449 satellites in the sample, there are 20 failure times and 429 censored times. The failure times are provided in Table A.1. The data is then treated with the Kaplan–Meier estimator, and the Kaplan–Meier plot of the GEO communication satellite reliability is obtained and shown in Figure A.1. Figure A.1 reads as follows. After a successful launch, for example, satellite reliability drops to approximately 98.6% after 2 years on orbit. More precisely, ˆ = 0.9858 R(t)

for

722 days ≤ t < 1122 days

that is, 1.977 years ≤ t < 3.072 years.

1.00 0.99 0.98

Reliability

0.97 0.96 0.95 0.94 0.93 0.92 0.91 0


Figure A.1 Kaplan–Meier plot of geosynchronous communication satellite reliability.


February 17, 2011

9:19


APPENDIX A

153

Table A.2 Tabular data for the Kaplan–Meier plot of geosynchronous communication satellite reliability in Figure A.1. Failure time ti (years) 0.0027 0.3368 0.3504 0.9966 1.9192 1.9767 3.0719 4.9199 5.9713 6.6502 6.7680 7.0554 7.0637 7.9863 8.0684 8.9473 9.4593 10.0862 11.4771 11.8385

ˆ i) R(t 0.9978 0.9955 0.9932 0.9908 0.9883 0.9858 0.9831 0.9801 0.9768 0.9733 0.9698 0.9661 0.9625 0.9585 0.9543 0.9496 0.9445 0.9387 0.9308 0.9224

Figure A.1 also shows for example that geosynchronous communication satellite reliability drops to approximately 97.7% after 6 years on orbit. Past 12 years, satellite reliability drops to 92.2%. The tabular data for Figure A.1 is given in Table A.2. One noticeable trend in Figure A.1 is the steeper decrease in satellite reliability after 6 years on orbit compared to the first 6 years. This trend is further discussed after the parametric fits are conducted.

A.1.2


As discussed in Chapter 2, the Kaplan–Meier estimator (Equation (2.10)) provides a maximum likelihood estimate of reliability, but does not inform about the dispersion ˆ i ). This dispersion is captured by the variance or standard deviation of around R(t the estimator, which is then used to derive the upper and lower bounds for, say, a 95% confidence interval (i.e., a 95% likelihood that the actual reliability will fall between the two calculated bounds, with the Kaplan–Meier analysis providing


154

February 17, 2011

9:19


APPENDIX A 1.00 0.99 0.98 0.97 Reliability

0.96 0.95 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0

Nonparametric estimation 95% confidence interval bounds 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time after successful orbit insertion (years)

Figure A.2 Geosynchronous communication satellite reliability with 95% confidence intervals.

the most likely estimate). The reader is referred to the details in Chapter 2 about calculations of confidence intervals. The results of the confidence interval analysis for the geosynchronous communication satellites are shown in Figure A.2. Vertical cuts across Figure A.2 read as follows, for example:

r Satellite reliability at t = 4 years will be between 97.1% and 99.6% with a 95% likelihood – these values constitute the lower and upper bounds of the 95% confidence interval at t = 4 years. In addition, the most likely estimate of satellite reliability at this point in time is Rˆ = 98.3%.

r Satellite reliability at t = 11 years will be between 91.0% and 96.7% with a 95% likelihood. In addition, the most likely estimate of satellite reliability at this point in time is Rˆ = 93.9%.

ˆ i ) increases with time. This can be seen Notice that the dispersion of R(ti ) around R(t in Figure A.2 by the growing gap between the Kaplan–Meier estimated reliability and the confidence interval curves, and is a result of the decreasing sample size with ˆ i ), which is time. The implication is that the nonparametric satellite reliability, R(t a maximum likelihood estimate derived by the Kaplan–Meier estimator, becomes less precise with increasing time on orbit. For example, after 12 years, the maximum likelihood estimate of satellite reliability is Rˆ = 92.2% with a 95% confidence interval dispersed over 7.2 percentage point, whereas at t = 2 years, Rˆ = 98.6% and the 95% confidence interval is dispersed over 2.3 percentage point. The tabular data for Figure A.2 is given in Table A.3.


February 17, 2011

9:19


APPENDIX A

155

Table A.3 Tabular data for the Kaplan–Meier plot of geosynchronous communication satellite reliability in Figure A.2. 95% confidence interval Failure time (years) 0.0027 0.3368 0.3504 0.9966 1.9192 1.9767 3.0719 4.9199 5.9713 6.6502 6.7680 7.0554 7.0637 7.9863 8.0684 8.9473 9.4593 10.0862 11.4771 11.8385

A.1.3

Lower bound

Upper bound

0.9934 0.9892 0.9855 0.9818 0.9781 0.9745 0.9707 0.9664 0.9618 0.9568 0.9519 0.9470 0.9422 0.9367 0.9311 0.9247 0.9178 0.9099 0.8983 0.8863

1.0000 1.0000 1.0000 0.9998 0.9985 0.9971 0.9955 0.9938 0.9919 0.9899 0.9877 0.9853 0.9829 0.9802 0.9775 0.9744 0.9711 0.9675 0.9632 0.9585

Parametric fits of geosynchronous communication satellite reliability

The Weibull plot of the nonparametric reliability for geosynchronous communication satellites is provided in Figure A.3. The reader is referred to Chapter 3 for details about Weibull plots. Several observations can be made regarding Figure A.3, the most important being that the data points are clearly not well aligned, thus making the single Weibull distribution an inaccurate model for the reliability of geosynchronous communication satellites. To highlight the trends of the nonparametric reliability results, two lines are superimposed on the empirical points in Figure A.3: line (1) is representative of an approximate fit done on the beginning of the sample, thus in the early times on orbit; line (2) approximates the trend of the points later on orbit. In addition, the slope of line (1) is less than one. Recall that the slope of a line on the Weibull plot provides the shape parameter of the associated Weibull distribution, which represents the failure behavior of the considered satellite population. Thus, a shape parameter less than one in line


156

February 17, 2011

9:19


APPENDIX A -2 -3

ln[-ln R]

-4 -5 -6 (1)

(2)

-7 -8 -7

-6

-5

-4

-3

-2

-1

0

1

2

3

ln(t)

Figure A.3 Weibull plot of the Kaplan–Meier estimated geosynchronous communication satellite reliability. (1) characterizes infant mortality in geosynchronous communication satellites. However, the slope of line (2) is greater than one, therefore its corresponding shape parameter is greater than one, indicating wear-out failure behavior on orbit. As a result, any averaged slope and corresponding shape parameter of a Weibull distribution would fail to capture the observed failure behavior of GEO communication satellites. Since the slopes vary from less than to greater than one, the failure rate of these satellites is nonmonotonic. Thus, Weibull distributions, as well as lognormal distributions, which have monotonic failure rates, can only capture one of the observed satellite failure trends, and fail to model the other. This discussion indicates the need for finite mixture distributions to accurately represent the nonparametric reliability of geosynchronous communication satellites. The MLE procedure was therefore used to derive the parameters of a 2-Weibull mixture distribution for these satellites’ reliability. The reader is referred to Chapter 3 for details about mixture distributions and the MLE procedure. The results are given in Table A.4. The resulting reliability function is given by R(t) = 0.9490 exp −

t 39 830.5

0.4458

t 4.6687 + 0.0510 exp − . 9.7

(A.1)

Table A.4 Maximum likelihood estimates of the parameters for the 2-Weibull mixture distribution for the reliability of geosynchronous communication satellites. Method MLE

β1

θ1 (years)

β2

θ2 (years)

α

0.4458

39 830.5

4.6772

9.7

0.9490


February 17, 2011

9:19


APPENDIX A

157

1.00 Nonparametric estimation 2-Weibull mixture (MLE)

0.99 0.98

Reliability

0.97 0.96 0.95 0.94 0.93 0.92 0.91 0


Figure A.4 Nonparametric results and 2-Weibull mixture distribution for the reliability of geosynchronous communication satellites. Note that the mixture distribution consists of one Weibull distribution with infant mortality (β 1 < 1) and one with wear-out failures (β 2 > 1). The parametric fit of the reliability of geosynchronous communication satellites is shown together with the nonparametric reliability results in Figure A.4. The failure rate derived from the 2-Weibull mixture distribution is shown in Figure A.5. The reader is referred to Chapter 4 for more details about failure rate. The y-axis is provided in a log-scale for readability purposes. The left panel in Figure A.5 provides a closer look at the failure rate over the early times on orbit, through the use of a log-scale on the x-axis. The result is a roller-coaster failure rate for geosynchronous communication satellites.

10

10

10

0

10 Failure rate (failures per year)


10

-1

-2

10

10

10

10

-3 -3

10

-2

-1

0

1

10 10 10 10 Time after successful orbit insertion (years)

10

1

0

-1

-2

-3

-4

0

1

2

3

4

5

6

7

8

9

10 11 12

Time after successful orbit insertion (years)

Figure A.5 Failure rate of the geosynchronous communication satellites from the 2-Weibull mixture distribution model.


158

February 17, 2011

9:19


APPENDIX A

Table A.5 Error between the nonparametric reliability and the parametric fits. Maximum error Model 2-Weibull mixture

Average error

(percentage point) 0.85

0.20

The quality of fit of this mixture distribution model is analyzed next. Visual inspection of Figure A.4 indicates that the 2-Weibull mixture distribution follows with good accuracy the nonparametric satellite reliability, capturing both the infant mortality trend and the wear-out failures (steep decline past 6 years). To quantitatively gauge this accuracy, the maximum and average errors between the nonparametric reliability (the benchmark) and the parametric model are calculated. The results are given in Table A.5 and provide a rough first measure of the quality of fit. Table A.5 indicates that, by using the Weibull mixture distribution, given by Equation (A.1), as a model for the reliability of geosynchronous communication satellites, one makes on average a 0.2 percentage point error with respect to the benchmark nonparametric reliability – the actual best estimate of the reliability of these systems. This represents a significant accuracy in reliability analysis. In addition to the calculation of the maximum and average errors, the results for the sum of square errors (SSE) for several parametric fits, single Weibull, 2-Weibull, and 3-Weibull mixture distributions,2 are provided in Figure A.6, and the box-plots of their residual errors are shown in Figure A.7. Figure A.6 shows a dramatic improvement in the accuracy of the parametric fit (reduction of the SSE) with the use of a mixture of distributions. However, limited improvement in accuracy is obtained by going from a 2-Weibull to a 3-Weibull mixture distribution. Figure A.7 shows the box-plots of the residuals or error dispersion between the parametric fit and the nonparametric results. Details about box-plots can be found in Chapter 3. Figure A.7 confirms the improvement in accuracy of the mixture distributions: their residuals have smaller spreads and dispersions than those of the single Weibull model (in terms of both their first and third quartiles, and their extreme values). Figure A.7 also shows that the single Weibull distribution is a biased reliability model of geosynchronous communication satellites, as evidenced by the nonsymmetric distribution of its residuals. Given these results, Part I of this appendix advances the 2-Weibull mixture distribution (Equation (A.1)) as the appropriate parametric reliability model of geosynchronous communication satellites.

2 The

single Weibull and 2-Weibull mixture distributions were derived using the MLE procedure. The 3-Weibull mixture distribution was derived with the nonlinear least squares method.


February 17, 2011

9:19


APPENDIX A

159

0.6 0.5

SSE

0.4 0.3 0.2 0.1 0.0 Single Weibull

2-Weibull mixture 3-Weibull mixture

Figure A.6 SSE of the different parametric fits.

Residuals (percentage point)

1.5 1.0 0.5 0.0 -0.5 -1.0 -1.5 Single Weibull

2-Weibull mixture

3-Weibull mixture

Figure A.7 Box-plots of the residuals between the parametric fits and the nonparametric reliability.

As an aside, and a final visual confirmation of the appropriateness of the 2-Weibull mixture distribution, the Weibull plot of Equation (A.1) is superimposed on that of the nonparametric reliability results, shown previously in Figure A.3. The result in Figure A.8 clearly shows a matching of the two identifiable local slopes, discussed previously, through the modeling of both infant mortality (β 1 < 1) and wear-out failures (β 2 > 1).


160

February 17, 2011

9:19


APPENDIX A -2 Nonparametric estimation 2-Weibull mixture

-3

ln[-ln R]

-4 -5 -6 -7 -8 -7

-6

-5

-4

-3

-2

-1

0

1

2

3

ln(t)

Figure A.8 Weibull plot of the nonparametric reliability and the 2-Weibull mixture distribution.

A.2

Part II: Subsystem anomalies and failures

In this section, the anomalies and failures of subsystems on board geosynchronous communication satellites are analyzed, and a health scorecard is developed summarizing for each subsystem its track record of on-orbit failure events. The previous population of 449 geosynchronous communication satellites analyzed is further narrowed down to: (1) satellites launched after January 1, 1995; and (2) satellites built by the major North American and European satellite manufacturers. Spin satellites were excluded from this study because they embody a relatively obsolete technology no longer representative of current spacecraft designs, and as such their failures are not representative of current spacecraft failures. The satellite platforms included here are the Boeing 601 Series, the Boeing 702 series, the EADS Eurostar 2000+, the EADS Eurostar 3000 series, the Lockheed Martin A2100A and A2100AX, the OSC STAR 2, the Spacebus 3000 and 4000 series, and the Loral 1300 series. A total of 166 satellites are analyzed in this second part of the appendix, and they have logged 924.9 years on orbit. In addition, 166 failure events were recorded during this time frame (January 1995 to October 2008): 60 minor, 104 major, and 2 fatal. Special cases: For the Boeing 601 and the Loral 1300, we include in our sample only the variations of these platforms launched after 1995. Thus, when the Boeing 601 series is mentioned, it does not include all Boeing 601 platforms, just the ones launched after January 1, 1995, namely, the Boeing 601 HP, the Boeing 601 Modified, and the Boeing 601 Goes-NP. Likewise, when the Loral 1300 series is mentioned in this work, it does not refer to all LS-1300 platforms, just the LS-1300 HL (extended), the LS-1300 SX, and the LS-1300 X.


February 17, 2011

9:19


APPENDIX A

161

Recall that the database used in this work recognizes the following subsystems (see Chapter 5 for details): 1. Gyro/sensor/reaction wheel (hereafter referred to as Gyro). 2. Thruster/fuel (Thruster). 3. Beam/antenna operation/deployment (Beam). 4. Control processor (CP). 5. Mechanisms/structures/thermal (Mechanisms). 6. Payload instrument/amplifier/on-board data/computer/transponder (Payload). 7. Battery/cell (Battery). 8. Electrical distribution (ED). 9. Solar array deployment (SAD). 10. Solar array operating (SAO). 11. Telemetry, tracking, and command (TTC). In addition to identifying the culprit subsystems, the database recognizes different classes of failure events. These classes are listed as follows in increasing order of failure severity:




r Class I: subsystem failure causing satellite retirement. This effectively means the total failure of the satellite due to a (dramatic) subsystem failure. For convenience and other structural reasons noted in Chapter 6, this work is restricted to three categories of failure events: minor (Class IV and Class III), major (Class II), and fatal (Class I).

A.2.1

Subsystem health scorecard

The concept of a health scorecard was developed to characterize in one snapshot the failure and anomaly track record of spacecraft subsystems on orbit. The scorecard contains both raw data collected from the database and results we computed by using different parameters, as we discuss next. Figure A.9 shows the health scorecard for the SAO subsystem, which is used here as an example to describe the general content and development of this tool. The scorecards for the remaining subsystems are provided


162

February 17, 2011

9:19


APPENDIX A

Solar Array Operating (SAO)

6 SAO Failure Histogram

1

Number of satellites that have experienced N failure events

Minor failure events: 3 Major failure events: 32 Fatal failure events: 1 Total failure events: 36 Percentage of SAO Failure Events out of Total Failure Events

21.7%

2

Percentage of SAO Failure Events out of Failure Events for a Given Severity Fatal

24 6

0

0

0

4

5

50.0%

Major

0

30.8%

Minor 3

136

3 1 2 N Failure Events

5.0%

SAO Failure Concentration Ratio

Average SAO Failure Events Per Time (FIT)

0.20 4,443 7 4 Average SAO Failure Events Per Time (FIT) for a Given Severity Fatal

123

Major Minor 5

3,950

a: Min

370

c: Median d: 75% e: Max 1.07

89% 8%

3.40

9.08

Average Time to SAO Failure Events in Years (Days) 5.83

Fatal

3%

Major Minor

b: 25%

0.02 (7) 0.46 (168)

SAO Failure Severity Percentages Fatal

Time to SAO Failure Events Boxplot in Years (Days)

Major Minor

2.34 0.68 (249)

Figure A.9 Health scorecard for the SAO subsystem.

at the end of this appendix. Recall that the observation window for this study extends from January 1995 to October 2008. The information given here reflects the failure and anomaly track record of subsystems of a particular population of communication satellites in geosynchronous orbit between these two dates. The health scorecards should be considered dynamic documents to be updated on a regular basis.


February 17, 2011

9:19


APPENDIX A

163

The top left corner of the scorecard provides the total number of failure events, as well as their breakdown by severity (minor, major, and fatal) for the subsystem under consideration. In the SAO example shown in Figure A.9, a total of 36 failure events occurred in this subsystem during the observation window: 3 minor, 32 major, and 1 fatal. The subsystem health scorecard is then divided into seven parts or boxes, described next. Box 1 provides the percentage of the subsystem failure events out of the total number of failure events recorded during the observation window, that is, out of the 166 failure events documented for the satellite population covered in this study. Box 1 shows the percentage of failures attributed to the subsystem under consideration. For the example in Figure A.9, Box 1 shows that 21.7% of all failure events were attributed to the SAO subsystem. This indicates a relatively high propensity of failure for the SAO subsystem compared to other subsystems, as will be discussed in the next section. Box 2 provides the extent of a subsystem’s contributions to all failure events of a given severity. For example, in the SAO case, 5.0% of all minor failure events (a total of 60) are attributed to the SAO subsystem, as well as 30.8% of all major failure events (a total of 104), and 50.0% of all fatal failure events (2). Box 3 provides the average failure rate of the subsystem under consideration (the ratio of total failure events per 1 billion hours, also known as FIT, logged on orbit for the satellite population here). For the example in Figure A.9, Box 3 shows that the SAO has an average failure rate of 4443 FIT. Box 4 expands on the result in Box 3 by distinguishing average failure rates for each severity of failure event. For example, Figure A.9 shows that the SAO has an average failure rate for fatal events of 123 FIT, an average failure rate for major anomalies of 3950 FIT, and an average failure rate for minor anomalies of 370 FIT. The three average failure rates in Box 4 add up to the average failure rate in Box 3 (except for rounding errors) as both boxes share the same denominator of time logged on orbit. Box 5 reflects the raw data provided in the top left corner of the scorecard, that is, the extent to which a subsystem has failed soft, hard, or fatally. More specifically, the percentages provided in Box 5 are calculated by dividing the number of failure events of a given severity for the subsystem under consideration by the total number of failure events of the said subsystem. For example, in the SAO case, 89% of all its failure events were major, thus indicating that the SAO tends to fail hard. Box 6 provides two important pieces of information: the subsystem’s failure histogram and its failure concentration ratio. The histogram shows the number of satellites that have experienced n failure events of the subsystem under consideration. For example, in the case shown in Figure A.9, 136 satellites have not experienced an SAO failure event, 24 satellites have experienced 1 SAO failure event, and 6 satellites have experienced 2 SAO failure events. In addition, the failure histogram indicates that no satellites experienced more than 2 SAO failure events in the sample. This histogram indicates how SAO failures are distributed among the 166 total satellite population covered in this work. The results in the histogram can be used to derive


164

February 17, 2011

9:19


APPENDIX A

several additional pieces of information. One important ratio included in the scorecard is the failure concentration ratio of the subsystem under consideration, shown in the lower part of Box 6. For example, in the case of the SAO, this ratio is the number of spacecraft that have experienced multiple SAO failure events divided by the total number of spacecraft that have experienced an SAO failure event. This ratio indicates the extent to which a specific subsystem has failed multiple times on a single spacecraft. The lower part of Box 6 shows that the SAO subsystem has a 0.20 failure concentration ratio, that is, 20% of the spacecraft that have experienced an SAO failure have experienced multiple ones. A high failure concentration ratio can raise several questions, for example, whether the subsystem failure events are dependent (e.g., common cause or cascading failures), or whether a specific spacecraft platform has a faulty subsystem with endemic problems. Box 7 provides a time dimension to the analysis, namely, the time to a failure event for the subsystem under consideration after insertion of the spacecraft into orbit. The results are displayed in a box-plot format (details about box-plots can be found in Chapter 3). For example, in the SAO case, Box 7 shows that the first failure event of this subsystem occurred 0.02 years (7 days) after launch, and the last SAO failure event occurred 9.08 years after launch. The median time to failure of the SAO is 1.07 years. The first and third quartiles are 0.46 years (168 days) and 3.40 years respectively. The convention adopted in this appendix is to display in parentheses the time to failure in days when it is less than 1 year. The box-plot in Figure A.9 indicates that of the satellites that experienced SAO failures, a large portion experienced these failure events early in their service life (50% within roughly the first year on orbit). In addition, the lower part of Box 7 provides the mean time to a failure event by severity. For example, in the SAO case, the mean time to a fatal failure event is 5.83 years, to a major failure event 2.34 years, and to a minor failure event 0.68 years (249 days), indicating that the more severe failure events of the SAO are likely to occur later in the life of the satellite. The health scorecards for the remaining subsystems are provided at the end of this appendix.

A.2.2

Comparative analysis of subsystem health scorecard

The health scorecards contain a wealth of information about the failure events of each spacecraft subsystem, and, as such, they constitute a good summary of each subsystem’s on-orbit failure track record. These scorecards can also be used for comparative analyses of failures and anomalies between different subsystems. Given the many parameters and statistics included in these scorecards, the possibilities for such analyses are extensive. In this section, a comparative analysis of the different subsystems is conducted based on a few select parameters in their health scorecards, namely, the “percentage of total failure events attributed to a particular subsystem,” the “failure severity percentages,” the “failure concentration ratio” versus “number of failure events” scatter plot, and the “time to failure box-plots,” as will be discussed next.


February 17, 2011

9:19


Percentage of total failure events attributed to a particular subsystem

APPENDIX A

165

30% 25.3%

25% 21.7%

20%

16.9% 13.9%

15% 10%

6.0%

5%

5.4% 3.6%

3.0%

1.8%

1.2%

0% Thruster Minor

7.8% Major 16.9% Fetal 0.6%

1.2%

0.0%

Mechanis Processor ms

Beam Elec Dist Unknown

SAD

0.6% 4.8%

0.6% 3.0%

0.6% 2.4%

0.0% 1.8%

0.0% 1.2%

0.6% 0.6%

0.0% 0.0%

0.0%

0.0%

0.0%

0.0%

0.0%

0.0%

0.0%

SAO

Payload

TT&C

Gyro

Battery

1.8% 19.3%

6.0% 10.8%

13.3% 0.6%

4.8% 1.2%

0.6%

0.0%

0.0%

0.0%

Subsystems

Figure A.10 Percentage of total failure events attributed to a particular subsystem. Figure A.10 collects in Box 1 information on all subsystems and compares the percentage of total failure events attributed to each subsystem. Each bar represents the total percentage of failure events and is further broken down by severity. Figure A.10 shows that the major culprits driving communication satellite failure events are the Thruster subsystem (25.3%), the SAO (21.7%), the Payload (16.9%), and the TTC (13.9%). Reliability improvements to these subsystems, through more extensive testing prior to launch, better part selection, or redundancy, will have a significant positive effect on communication satellites. Conversely, Figure A.10 shows that at the other end of the failure propensity spectrum, the Mechanisms and SAD subsystems together contributed 2.4% of the total failure events, and the particularly robust CP subsystem contributed no failure events to the satellite population in this part of the appendix. In addition to revealing the most problematic subsystems, in terms of frequency of failure events, Figure A.10 yields valuable information regarding the severity of failure events by subsystem. Notice for example that while the Gyro and the Battery subsystems have roughly the same percentage contributions to the total number of failure events (6.0% and 5.4% respectively), they differ significantly when their failure events are broken down by severity: the Gyro more often fails “soft” whereas the Battery more often fails “hard” (Class II failure) and thus more severely affects the operation (degradation) of the satellite. Figure A.11 collects information in Box 5 on all subsystems and displays the severity percentages of failures of each subsystem. The results are displayed in decreasing order of the percentage of major failure events. Figure A.11 emphasizes the severity of failure information whereas Figure A.10 emphasizes the frequency of failure events per subsystem. For example, Figure A.11 shows that the TTC subsystem more often fails “soft” and 96% of its failures were minor anomalies, whereas the Battery or the SAO subsystems more often fail “hard” and 89% of their failures were major anomalies. Figure A.11 provides a more complex perspective on


166

February 17, 2011

9:19


APPENDIX A 100 %=2 100 %=36

100 %=9

100 %=6

100 %=5 100 %=42 100 %=28 100 %=2

100 %=10 100 %=23

Failure Severity Percentage

100 %=3

Unknown Minor 0% Major 100% Fatal 0%

SAD

SAO

0% 100% 0%

8% 89% 3%

Battery/ Beam/ Electrical Thruster/ Payload Mechanisms Cell Antenna Dist Fuel 11% 20% 31% 35% 50% 17% 89% 80% 67% 64% 50% 83% 0% 0% 2% 0% 0% 0%

Gyro

TTC

80% 20% 0%

96% 4% 0%

100 %=0 Control Processor 0% 0% 0%

Figure A.11 Failure severity percentages. some subsystems, given the three classes of failure events considered (minor, major, and fatal). Figure A.12 collects information in Box 7 on all subsystems, the box-plots of the time to failure events, and displays them in decreasing order of the median time to failure. The figure shows both the comparative dispersion (whiskers) of the time to failure between the different subsystems as well as their concentration (first and third quartiles). For example, the time to a failure event of the Battery is confined to the first 2 years on orbit, whereas it is spread over the first 10 years for the Gyro. In addition, Figure A.12 shows that of the 10 subsystems considered, eight have medians for time to a failure event of less than 2 years, and all have failure events recorded in roughly

Time to Subsystem Failure Event (Years)

12.00

10.00 8.00

6.00

4.00

2.00 0.00 Gyro

TTC

Thruster

Payload

SAO

Battery Mechanisms Elecrical Beam Distribution

SAD

Maximum

10.87

6.11

7.64

8.53

9.08

2.00

1.08

10.31

2.42

0.18 (64)

3rd Quartile

7.02

4.66

3.30

3.90

3.40

1.14

–

5.18

0.32 (117)

–

Median

4.02

3.18

1.78

1.36

1.07

0.99 (360)

–

0.48 (174)

0.13 (47)

–

1st Quartile

2.17

1.06

0.60 (220)

0.50 (182)

0.46 (168)

0.33 (121)

–

0.38 (138)

0.09 (34)

–

Minimum

0.19 (68)

0.03 (12)

0.06 (22)

0.01 (2)

0.02 (7)

0.00 (1)

0.15 (55)

0.18 (64)

0.03 (10)

0.03 (12)

Figure A.12 Time to subsystem failure event in years (days).


February 17, 2011

9:19


APPENDIX A

167

45 Thruster

Number of failure events

40 SAO

35 30

Payload

25

TTC

20 15 Gyro

10 5

Battery/Cell Beam

Electrical Disc Unknown Mechanisms, SAD Controller Processor

0 0

0.05

0.1

0.15

0.3 0.35 0.2 0.25 Failure Concentration Ratio

0.4

0.45

0.5

Figure A.13 Number of failure events versus failure concentration ratio. the first two months on orbit. This indicates a propensity to infant mortality (or infant anomaly) of all subsystems, and an opportunity for satellite manufacturers to weed out these failures through improved testing procedures prior to launch. It should be noted that the results shown in Figure A.12, while indicating infant mortality, do not exclude “wear-out” failures since the population of satellites studied here is relatively young, with an average age of 5.57 years (recall that the population consists of geosynchronous platforms launched between January 1995 and October 2008). Consequently, wear-out failures may still manifest themselves over time as this population of satellites ages (the result would be a shift upward of the “boxes” in Figure A.12). Figure A.13 is a scatter plot of the number of failure events per subsystem versus the failure concentration ratio, defined earlier as the ratio of satellites that have experienced multiple failure events to the satellites that have experienced at least one failure event, by subsystem. Five subsystems – the Gyro, SAD, ED, Mechanisms, and CP – plus the Unknown category did not experience multiple failure events on a single satellite; they are single-failure per satellite subsystems. Figure A.13 shows that while the SAO and Thruster subsystems are major contributors to satellite failure events (42 and 36 failure events respectively), their failure concentration ratios are significantly different (0.46 and 0.20 respectively), which reflects the fact that the large number of failure events of the Thruster are generated by a few bad satellites. In addition, compare for example the Battery and TTC subsystems, both having roughly similar failure concentration ratios (0.29 and 0.33 respectively), yet clearly different propensities to failure. The implication is that, although there are (satellite) repeat offenders with respect to failures of these two subsystems, current battery technology is likely intrinsically more robust than that of TTC. A similar comment can be made about the Beam and the SAO subsystems. The health scorecards for all the satellite subsystems are provided next.


168

February 17, 2011

9:19

APPENDIX A 6

Gyro

Gyro Failure Histogram

Minor failure events: 8 Major failure events: 2 Fatal failure events: 0 Total failure events: 10

156 Number of satellites that have experienced N failure events

1

Percentage of Gyro Failure Events out of Total Failure Events

6.0%

2 Percentage of Gyro Failure Events out of Failure Events for a Given Severity Fatal

0.0%

Major

10 0 0

1.9%

Minor 3


1

0

2 3 N Failure Events

0

0

4

5

13.3%

Gyro Failure Concentration Ratio

Average Gyro Failure Events Per Time (FIT)

No satellites experienced multiple gyro failures 1,234 7 4 Average Gyro Failure Events Per Time (FIT) for a Given Severity

Time to Gyro Failure Events Boxplot in Years (Days)

Fatal 0 Major

247

Minor 5

a: Min 987

0.19 (68)

Gyro Failure Severity Percentages

Minor

2.17

4.02

d: 75% e: Max 7.02

10.87

Average Time to Gyro Failure Events in Years (Days) Fatal 0.00

Fatal 0% Major

b: 25% c: Median

Major

20% 80%

Minor

5.97 4.17


February 17, 2011

9:19


APPENDIX A 6

Thruster/Fuel

Thruster Failure Histogram


Minor failure events: 13 Major failure events: 28 Fatal failure events: 1 Total failure events: 42 1

Percentage of Thruster Failure Events out of Total Failure Events

25.3%

2

Percentage of Thruster Failure Events out of Failure Events for a Given Severity Fatal

50.0%

Major

138

15

12 1

0

26.9%

Minor

169

1


0

0

4

5

21.7%

3

Thruster Failure Concentration Ratio

Average Thruster Failure Events Per Time (FIT)

0.46 5,184 7 4 Average Thruster Failure Events Per Time (FIT) for a Given Severity Fatal

123

Major

3,456

Minor 5

1,605

67% 31%

b: 25% c: Median d: 75% e: Max 1.78

3.30

7.64

Average Time to Thruster Failure Events in Years (Days) Fatal

2%

Major Minor

a: Min

0.06 (22) 0.60 (220)

Thruster Failure Severity Percentages Fatal

Time to Thruster Failure Events Boxplot in Years (Days)

1.00

Major Minor

2.63 1.59


170

February 17, 2011

9:19

APPENDIX A 6

Beam/Antenna

Beam Failure Histogram



1


Percentage of Beam Failure Events out of Total Failure Events

3.6%

2 Percentage of Beam Failure Events out of Failure Events for a Given Severity Fatal

0.0%

Major

0

4.8%

Minor

4 1

1

0


0

0

4

5

1.7%

3

Beam Failure Concentration Ratio

Average Beam Failure Events Per Time (FIT)

0.20 741 7 4 Average Beam Failure Events Per Time (FIT) for a Given Severity

Time to Beam Failure Events Boxplot in Years (Days)

Fatal 0 Major Minor 5

617 123

Beam Failure Severity Percentages

b: 25% c: Median d: 75% e: Max

Average Time to Beam Failure Events in Years (Days) Fatal 0.00

Fatal 0% Major Minor

a: Min

0.03 (10) 0.09 (34) 0.13 (47) 0.32 (117) 2.42

83% 17%

Major Minor

0.15 (55) 2.42


February 17, 2011

9:19


APPENDIX A 6

Mechanisms/Structures

Mechanisms Failure Histogram



1

Percentage of Mechanisms Failure Events out of Total Failure Events

1.2%

2 Percentage of Mechanisms Failure Events out of Failure Events for a Given Severity Fatal

0.0%

Major Minor

171

2 0

1.0%

1

0

0


0

0

4

5

1.7%

3

Mechanisms Failure Concentration Ratio

Average Mechanisms Failure Events Per Time (FIT)

No satellites experienced multiple mechanisms failures

247

7 4 Average Mechanisms Failure Events Per Time (FIT) for a Given Severity

Time to Mechanisms Failure Events Boxplot in Years (Days)

Fatal 0

5

Major

123

Minor

123

Mechanisms Failure Severity Percentages

a: Min

b: 25% c: Median d: 75% e: Max

0.15 (55)

-

-

-

1.08

Average Time to Mechanisms Failure Events in Years (Days) Fatal 0.00

Fatal 0% Major

50%

Minor

50%

Major Minor

0.15 (55) 1.08


6 Payload Failure Histogram


151

Percentage of Payload Failure Events out of Total Failure Events

16.9%

2 Percentage of Payload Failure Events out of Failure Events for a Given Severity Fatal

3


APPENDIX A

Payload/Transponder

1

9:19


172

February 17, 2011

10 2

2

Major

17.3%

Minor

16.7%

0

1

2

0

0

0

1

3 4 5 6 N Failure Events

7

8

0.0%

0

Payload Failure Concentration Ratio

Average Payload Failure Events Per Time (FIT)

0.33 3,456

7

4 Average Payload Failure Events Per Time (FIT) for a Given Severity

Time to Payload Failure Events Boxplot in Years (Days)


2,222 1,234

Fatal 0%

Minor

64% 36%

b: 25% c: Median

0.01(2) 0.50(182)

Payload Failure Severity Percentages

Major

a: Min

1.36

d: 75% e: Max 3.90

8.53

Average Time to Payload Failure Events in Years (Days) Fatal 0.00 2.89

Major Minor

1.56


February 17, 2011

9:19


APPENDIX A 6

Battery/Cell

Battery Failure Histogram



Percentage of Battery Failure Events out of Total Failure Events

5.4%

2 Percentage of Battery Failure Events out of Failure Events for a Given Severity Fatal

0.0%

Major Minor 3

173

159

5 0

7.7%

1

2

0


0

0

4

5

1.7%

Battery Failure Concentration Ratio

Average Battery Failure Events Per Time (FIT)

0.29 1,111 7 4 Average Battery Failure Events Per Time (FIT) for a Given Severity

Time to Battery Failure Events Boxplot in Years (Days)


987

a: Min

123

Fatal 0%

Minor

89% 11%

c: Median d: 75% e: Max

0.00 (1) 0.33 (121) 0.99 (360)

Battery Failure Severity Percentages

Major

b: 25%

1.14

2.00

Average Time to Battery Failure Events in Years (Days) Fatal 0.00 Major Minor 0.00(1)

0.99 (360)


Minor failure events: 1 Major failure events: 4 Fatal failure events: 0 Total failure events: 5 Percentage of ED Failure Events out of Total Failure Events

Percentage of ED Failure Events out of Failure Events for a Given Severity Fatal

0.0%

Major

ED Failure Histogram

5 0

3.8%

Minor 3

6

161

3.0%

2


APPENDIX A

Electrical Distribution (ED)

1

9:19


174

February 17, 2011

1

0

0


0

0

4

5

1.7%

ED Failure Concentration Ratio

Average ED Failure Events Per Time (FIT)

No satellites experienced multiple ED failures

617

7

Time to ED Failure Events Boxplot in Years (Days)

4 Average ED Failure Events Per Time (FIT) for a Given Severity Fatal 0 Major

494

Minor 5

123

ED Failure Severity Percentages

b: 25%


10.31

Average Time to ED Failure Events in Years (Days) Fatal 0.00

Fatal 0% Major Minor

a: Min

0.18 (64) 0.38 (138) 0.48 (174)

80% 20%

Major Minor

4.03 0.38 (138)


February 17, 2011

9:19


APPENDIX A

Solar Array Deployment (SAD) Minor failure events: 0 Major failure events: 2 Fatal failure events: 0 Total failure events: 2

1.2%

2 Percentage of SAD Failure Events out of Failure Events for a Given Severity 0.0%

Major Minor 3

SAD Failure Histogram 164

Percentage of SAD Failure Events out of Total Failure Events

Fatal

6


1

175

2 0

1.9%

1

0


0

0

0

4

5

0.0%

SAD Failure Concentration Ratio

Average SAD Failure Events Per Time (FIT)

No satellites experienced multiple SAD failures 247 7 4 Average SAD Failure Events Per Time (FIT) for a Given Severity

Time to SAD Failure Events Boxplot in Years (Days)

Fatal 0 Major

247

Minor 0 5

SAD Failure Severity Percentages Fatal 0% Major Minor 0%

a: Min 0.03 (12)

100%

b: 25% c: Median –

–

d: 75% e: Max –

0.18 (64)

Average Time to SAD Failure Events in Years (Days) Fatal 0.00 Major Minor 0.00

0.10 (38)


176

February 17, 2011

9:19

APPENDIX A

Solar Array Operating (SAO)

6 SAO Failure Histogram



Percentage of SAO Failure Events out of Total Failure Events

21.7%

2

Percentage of SAO Failure Events out of Failure Events for a Given Severity Fatal

136

24 6

0

0

0

4

5

50.0%

Major

0

30.8%

Minor 3


3 1 2 N Failure Events

5.0%


Average SAO Failure Events Per Time (FIT)

0.20 4,443 7 4 Average SAO Failure Events Per Time (FIT) for a Given Severity

Time to SAO Failure Events Boxplot in Years (Days)


3,950

a: Min

370

Minor

89% 8%

1.07

3.40

9.08

Average Time to SAO Failure Events in Years (Days) 5.83

Fatal

3%

Major

c: Median d: 75% e: Max

0.02 (7) 0.46 (168)

SAO Failure Severity Percentages Fatal

b: 25%

Major Minor

2.34 0.68 (249)


February 17, 2011

9:19


APPENDIX A 6

Telemetery, Tracking, and Command (TTC)

TTC Failure Histogram 151 Number of satellites that have experienced N failure events


Percentage of TTC Failure Events out of Total Failure Events

13.9%

2

Percentage of TTC Failure Events out of Failure Events for a Given Severity Fatal Major

10 4

0

0

1

4

5

0.0% 3 1 2 N Failure Events

0

1.0%

Minor 3

177

36.7%


Average TTC Failure Events Per Time (FIT)

0.33 2,839 7 4 Average TTC Failure Events Per Time (FIT) for a Given Severity

Time to TTC Failure Events Boxplot in Years (Days)

Fatal 0 Major

123

Minor 5

2,715

TTC Failure Severity Percentages

Minor

b: 25% 1.03


4.66

6.11

Average Time to TTC Failure Events in Years (Days) Fatal 0.00

Fatal 0% Major

a: Min 0.03 (12)

Major

4% 96%

Minor

3.18 2.95


February 17, 2011

10:20


Appendix B

Electrical power subsystem: comparative analysis of failure events in LEO and GEO1 B.1

Introduction

The electrical power subsystem (EPS) on board spacecraft, as seen in Chapter 6, is a major driver of on-orbit anomalies and failures. In the database used for this work (details can be found in Chapter 2), the EPS comprises the Battery, ED (electrical distribution subsystem), and the SAD (solar array deployment) and SAO (solar array operating). The objective of this appendix is to investigate the difference in EPS failure behavior, if any, in LEO (Low Earth Orbit) versus GEO (Geosynchronous Orbit). LEOs are typically orbits below 2000 km, and GEOs are circular or near circular orbits with apogee and perigee at about 36 000 km. Specializing the EPS anomaly and failure data by orbit and investigating potential differences in this subsystem’s failure behavior were motivated by the recognition that the power/load cycles and the space environment experienced by the EPS are significantly different in LEO and GEO. For example, spacecraft in LEO experience significantly more eclipses, thus significantly more charging/discharging cycles of their batteries, than those in GEO. In addition, spacecraft in GEO are exposed to a harsher radiation environment than their counterparts in LEO (leading for example to more stresses on the GEO solar arrays). It is then fair to ask if these differences in

1 This appendix was written in collaboration with So Young Kim. It is based on an article presented at the 2011 IEEE Aerospace Conference (Kim et al., 2011).



180

February 17, 2011

10:20


APPENDIX B

operational conditions and environment result in differences in failure behaviors of the ESP in LEO and GEO. This appendix addresses this question.

B.2

Database, sample analyzed, and classes of failure events

The sample of spacecraft analyzed here was collected from the SpaceTrak database described in Chapter 2. The sample consists of 937 Earth-orbiting satellites successfully launched between January 1990 and October 2008: 493 satellites in GEO and 444 satellites in LEO. Of the 1584 spacecraft previously analyzed, the spacecraft in medium Earth orbit (MEO) and those in a highly elliptical orbit (HEO) were removed from consideration. In addition, small spacecraft (m ≤ 500 kg) were excluded from this analysis. These spacecraft are primarily found in LEO, and because they may be single string or have peculiar design features (see Chapter 4 for details), they can bias the proposed comparative analysis. Since small spacecraft are predominant in LEO, their exclusion allows better control of the homogeneity of the EPS technology analyzed and, as a result, differences in failure behavior, if any, of the EPS are more likely to be the result of differences in operational conditions and environment in LEO and GEO, rather than the confounding effect of design and technology differences through the preponderance of small spacecraft in LEO and their absence in GEO. The EPS in the database comprises the following elements, hereafter referred to as subsystems for convenience:

r Battery/cell subsystem (referred to as Battery). r Electrical distribution subsystem. r Solar array deployment. r Solar array operating. The statistical analysis is enabled by, and confined to, the classes of (partial) failures identified in the database. These classes are listed next in increasing order of failure severity:




r Class I: subsystem failure causing satellite retirements. This effectively means the total failure of the satellite due to a (dramatic) subsystem failure. Class IV and Class III failure events are lumped together hereafter, first because both classes of events have a minor impact on the system or subsystem functionality,


February 17, 2011

10:20


APPENDIX B

181

and second because the database does not contain information on the presence of subsystem redundancy, or lack thereof, on board spacecraft. The absence of this information precludes meaningful interpretation of the results of the separate statistical analysis of Class IV and Class III events.

B.3

Brief literature review

Designing high reliability in satellites is subordinated to the knowledge and understanding of spacecraft failure behavior and mechanisms. Several parameters can impact or be correlated with varying spacecraft reliability, such as mass, complexity, and orbit. At the subsystem level, the spacecraft orbit is a prime suspect for affecting the anomalies and failures of the EPS. As noted previously, there are significant differences in environmental conditions and operational constraints on the EPS in LEO and in GEO, as briefly reviewed next.

B.3.1

Space environment in LEO and GEO

The differences in space environment in LEO and GEO pertain to the upper atmosphere, the plasma, and the radiation environment. A rarefied atmosphere is present in LEO, and it can create atmospheric drag and heating phenomena on spacecraft. The presence of highly reactive elements can also have “chemically corrosive effects” on spacecraft constituent surfaces (Waltersheid, 1999). An example of such corrosive elements is atomic oxygen, typically found between altitudes of 200 and 600 km, which “can react with thin organic films, advanced composites, and metalized surfaces, resulting in degraded sensor performance” (Waltersheid 1999, citing Visentine 1988). The EPS, through its solar panels, is directly affected by this type of corrosion and can suffer anomalies and failures as a consequence of it. Plasma in the Earth’s environment is present in both LEO and GEO, and it can charge spacecraft to high negative voltages (Koons, 1999). Spacecraft charging in turn can lead to electrostatic discharge (ESD), or arcing phenomenon, which may result in surface damage and induce electromagnetic interference or disruption to spacecraft electronics (Koons 1999, citing Robinson 1989). This phenomenon is of interest here as the plasma characteristics differ between LEO and GEO, and thus can lead to different arcing conditions and potential damage. For example, Bedingfield et al. (1996) show that LEO has a dense low-energy plasma, while GEO has a scarce high-energy plasma, thus creating different conditions for arcing and ESD in LEO and GEO (Ferguson et al., 1999). The damage created by ESD can severely degrade the performance of the spacecraft’s solar array, and much interest has been shown in understanding, modeling, and simulating arcing on orbit (Hoeber et al., 1998; Katz et al., 1998; Snyder, 1982; Frezet et al., 1989). Simulation results tend to show that arcing may be more damaging in GEO than in LEO (Vayner et al., 2007). Radiation effects on spacecraft might also be different in LEO and GEO. For example, the Van Allen belts (Schulz and Vampola, 1999) create to some extent a shielding effect for spacecraft in LEO against solar particle events (SPE) and galactic cosmic rays (GCRs) (Gorney, 1999; Blake, 1999). These radiations, SPEs and GCRs,


182

February 17, 2011

10:20


APPENDIX B

can affect spacecraft solar panels and electronics, and lead to single-event upset (SEU) or burn-out (SEB) with different damaging effects.

B.3.2

Operational constraints on the EPS in LEO and GEO

In addition to differences in environmental conditions in LEO and GEO, the operational constraints and requirements on the EPS are significantly different in LEO and GEO, and as such they may cause different EPS failure behaviors. For example, spacecraft in LEO, as noted previously, experience significantly more eclipses, thus significantly more charging/discharging cycles of their batteries. Typically, a spacecraft in LEO can experience an eclipse once per orbit or about 15 eclipse periods per day, each lasting no more than (roughly) half an hour in duration, while GEO spacecraft only encounter “two 45-day eclipse periods per year with eclipses lasting no more than 72 minutes each day” (McDermott, 1999). As a result, the number of charging/discharging cycles of spacecraft batteries in LEO and GEO are significantly different, and so is their depth of discharge (DOD). The extensive and frequent switching between power sources (Surampudi et al., 2006) in LEO, as well as the thermal cycling in and out of eclipses, may result in a variety of different stresses on the spacecraft in general, and the EPS in particular. The result of this discussion is that there is a justifiable possibility that spacecraft EPS exhibit different anomaly and failure behaviors in LEO and GEO. The next section reviews previous reliability results and multi-state failure analysis of the EPS. It is followed in Section B.5 by a comparative analysis of EPS failure behavior in LEO and GEO.

B.4

Reliability and multi-state failure analyses of the EPS

Previous chapters contain reliability and multi-state failure analyses of the EPS (Chapter 5 and Chapter 7 respectively). The population of spacecraft analyzed consisted of 1584 Earth-orbiting spacecraft successfully launched between January 1990 and October 2008 without distinction of orbit. The reliability analysis at the system (spacecraft) level with orbit as a covariate was conducted in Chapter 4. As a reminder, reliability analysis is concerned with Class I failures, that is, the total loss of a spacecraft due to the failure of a subsystem. Figure B.1 shows the nonparametric reliability results of the four subsystems encompassed by the EPS. Figure B.1 reads as follows. For example, after a successful launch, the reliability of the ED drops to approximately 99.4% after 6 years on orbit (best estimate). In addition, the reliability of this subsystem will fall between 99.0% and 99.9% with a 95% likelihood (or confidence interval) at this point in time. This same “reading ˆ and confidence interval applies to all grid” regarding the estimated reliability R(t) the other subsystems. Notice the particular nonparametric reliability of the SAD (a constant), which is due to the one-shot nature of this “subsystem” (or, more precisely, this phase of the solar array subsystem). Weibull fits of the nonparametric reliability results can be found in Chapter 5.


February 17, 2011

10:20


APPENDIX B Battery

183

ED

1.000

1.000 0.998

0.995


Reliability

0.990 0.985 0.980

0.992 0.990 0.988

0.975

0.986 0.970


0.984



SAD

SAO

1.0000

1.000 0.998

0.9995


Reliability

0.994 0.9985 0.9980 0.9975

0.992 0.990 0.988 0.986

0.9970

0.984

0.9965

0.982



Figure B.1 Nonparametric reliability of the EPS (its elements). As discussed in Chapter 7, a better understanding of the failure behavior of spacecraft subsystems is gained by investigating multi-state failure analyses beyond the traditional binary-state concept of reliability. Multi-state failure analysis introduces “degraded states” or partial failures, and thus provides more insights through finer resolution into the degradation behavior of an item and its progression toward complete failure. Building on the classes of events noted previously in Section B.2, the following four subsystem states were introduced, numbered from 4 to 1 in increasing order of failure severity or functionality degradation:

r State 4: Fully operational. r State 3: Class IV and Class III partial failure (hereafter referred to as “minor” anomaly/degradation).

r State 2: Class II partial failure (referred to as “major” anomaly/degradation). r State 1: Class I total failure (referred to as “fatal”). The failure state diagram for each subsystem is shown in Figure B.2. The multi-state failure analysis of the Battery, ED, and SAO subsystems yielded the following results (see Figures B.3–B.8).


184

February 17, 2011

10:20


APPENDIX B Fully operational

S4

Class IV–Class III S3

S2 Class I

Minor anomaly/degradation

Class II Major anomaly/degradation

S1 Total failure

Figure B.2 Multi-state and transition diagram for spacecraft subsystem. Battery 0.014


0.995

0.012 0.010 Probability

Probability

0.990 0.985 0.980

0.008 0.006 0.004

0.975 0.970


0.002

0

0.000


Figure B.3 Reliability and probability of being fully operational for Battery.


Figure B.4 Degraded states probabilities for Battery.

ED 0.012


0.010

0.995


0.985

Probability

Probability

0.008 0.990

0.006 0.004

0.980

0.002



Figure B.5 Reliability and probability of being fully operational for ED.

Figure B.6 Degraded states probabilities for ED.


February 17, 2011

10:20


APPENDIX B

185

SAO 1.00

0.08

0.99

0.07

0.98

0.06


0.96 0.95


0.94 0.93

Probability

Probability

0.97 0.05 0.04 0.03 0.02

0.92 0.01

0.91 0.90 0


Figure B.7 Reliability and probability of being fully operational for SAO.

0.00 0


Figure B.8 Degraded states probabilities for SAO.

The figures in the left column (Figures B.3, B.5, and B.7) show, for a given subsystem, its reliability curve along with its probability of being in state 4, that is, being fully operational. The reliability curve, or survivor function, represents the probability of the subsystem not being in the failed state 1. The distinction between these two curves is made possible in multi-state failure analysis by accounting for partial failures. For example, Figure B.7 shows that although the reliability of the SAO is approximately 99% after 15 years on orbit, its probability of being in a fully operational state (S4) is only 91% after 15 years on orbit. The gap between these two curves is filled by the probability of the subsystem being in any one of the partially degraded states, which are seen in the figures in the right column (Figures B.4, B.6, and B.8). For example, Figure B.8 shows that the SAO is 7% likely to be in state 2 (major anomaly/degradation) after 15 years on orbit. The next section breaks down the spacecraft population by orbit, and analyzes the differences in EPS failure events in LEO and GEO.

B.5

Comparative analysis of the EPS failure behavior in LEO and GEO

In this section, the data for EPS anomaly and failure is specialized by orbit (LEO and GEO). Recall that the population analyzed consists of 937 Earth-orbiting satellites successfully launched between January 1990 and October 2008, with 493 satellites in GEO and 444 satellites in LEO. In the following, the distribution of EPS failure events (extent, frequency, and severity) in LEO and GEO is analyzed and compared. Table B.1 gives a snapshot of various statistics of the sample. One important difference in Table B.1 is immediately visible: namely, the roughly order of magnitude more EPS failure events in GEO than in LEO. First, note the ratio of anomaly and failure events to sample size, which can be construed as the average


186

February 17, 2011

10:20


APPENDIX B

Table B.1 EPS anomaly and failure events in LEO and GEO.

Spacecraft sample size Total time logged on orbit (years) Number of EPS failure events Ratio of events to sample size Average failure event per time (FIT∗ ) ∗

LEO

GEO

447 2378 17 0.0380 816

493 4015 126 0.2556 3580

FIT = failure per 109 hours of operation (here, on orbit).

number of failure events per spacecraft: a spacecraft in LEO experienced on average 0.0380 EPS failure events, whereas its counterpart in GEO experienced 0.2556 EPS failure events. This result does not include information about the temporal dimension of the occurrence of failures. Such information is important, in particular because the average age of spacecraft in LEO is approximately 5.4 years and in GEO 8.1 years. In other words, spacecraft in LEO (analyzed here) are on average younger than their counterparts in GEO, and as such they have less time on orbit to experience EPS failure events. This age difference is accounted for by calculating an average EPS failure rate, shown in the last column of Table B.1: 816 FIT in LEO versus 3580 FIT in GEO. The result indicates roughly four times more EPS failure events per time in GEO than in LEO. Figure B.9 displays these results graphically and allows a quick comparative assessment of the distinctiveness of failure propensity of the EPS in LEO and GEO. Recall that in the database used for this work, the EPS comprises the Battery, ED, and the SAD and SAO. Figure B.10 illustrates the percentage contribution from each

Average failure event per time (FIT) 0

500

LEO

1000

1500

2000

2500

3000

3500

816

GEO

LEO

3580

0.0380

GEO 0.00

0.2556

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Ratio of number of events to sample size

Figure B.9 EPS anomaly and failure behavior in LEO and GEO.


February 17, 2011

10:20


APPENDIX B LEO satellites

187

GEO satellites Battery / Cell 12% Battery / Cell 29%

SAO 36%


SAD 5%


SAD 6%

SAO 69%

Total = 17

Total = 126

Figure B.10 EPS subsystem anomaly and failure event repartition. subsystem to the total number of EPS anomaly and failure events. The important features in Figure B.10 are as follows:

r The EPS failure events in GEO are predominantly SAO events (69%). r In LEO, EPS failure events are roughly equally distributed, or caused by, the Battery, SAO, and ED.

r SAD accounts for the smallest part of EPS failure events in LEO and GEO (6% and 5% respectively). The multiplicity of failure events on board a single spacecraft was a minor concern in this study, as five spacecraft experienced more than two anomalies on orbit in GEO and only one in LEO. In addition the ratio of spacecraft with one and two EPS failure events was the same and equal to 2.7 in LEO and GEO. Table B.2 provides a breakdown of various statistics of failure events for the four subsystems that constitute the EPS. Figure B.11 displays these results (Table B.2) graphically and allows a quick comparative assessment of the distinctiveness of failure behavior of each subsystem in LEO and GEO. The important results in Table B.2 and Figure B.11 are the following:

r The failure events of the Battery and ED are two to three times more extensive and frequent in GEO than in LEO.

r The SAD failure events are roughly five times more extensive in GEO than in LEO.

r And, more importantly, the failure events of the SAO are an order of magnitude more extensive and frequent in GEO than in LEO.


188

February 17, 2011

10:20


APPENDIX B

Table B.2 EPS breakdown anomaly and failure events. LEO

GEO

Battery Number of failure events Ratio of failure events to sample size Average failure event per time (FIT)

5 0.0112 240

15 0.0304 426

ED Number of events Ratio of failure events to sample size Average failure event per time (FIT)

5 0.0112 240

18 0.0365 511

SAD Number of events Ratio of failure events to sample size Average failure event per time (FIT)

1 0.0022 NA

6 0.0122 NA

SAO Number of events Ratio of failure events to sample size Average failure event per time (FIT)

6 0.0134 288

87 0.1765 2472

Given the sample size considered here, 493 satellites in GEO and 444 satellites in LEO, the discussion that follows explores whether these differences in EPS failure behavior are statistically significant (p < 0.05). Two nonparametric hypothesis tests were used: the log-rank test and the Wilcoxon test (details about these tests can be found in Lawless (2003) and Kvam and Vidakovic (2007)). Because the log-rank test was not applicable to the Battery subsystem, the Wilcoxon test was used as a backup test. Using the recorded times of anomalies and failures, these tests assess the difference in survival of two samples and indicate if this difference is statistically significant. Table B.3 provides the resulting p-values from the two tests.2 As shown in Table B.3, the overall EPS shows extremely small p-values (p 0.05) from both tests, which indicates that the difference in its failure behavior in LEO and GEO is indeed statistically significant. In addition, Table B.3 shows that the difference in failure behavior of the Battery and the SAO, in LEO and GEO, is also statistically significant (p < 0.05). The results for the ED subsystem are inconclusive. The previous paragraphs analyzed the EPS failure events without considering the severity of these events. This subsection investigates the severity distribution of 2 The

SAD is a one-shot subsystem, and, as such, these tests do not apply to it.

0.10

0.15

1500

0.20

2000

0.00

GEO 0.05

0.10

0.15

0.20

0.25

0.00

0

0.05

0.0365

511

0.10

1000

0.15

1500

0.20

2000

0.05

0.10

1000

0.15

1500

0.20

0.1765

2000


0.0134

288

500

Average failure event per time (FIT)

SAO


0.0112

240

500

0.25

2472

2500

0.25

2500

APPENDIX B

Figure B.11 Failure and anomaly behavior in LEO and GEO for each EPS subsystem.


0.0122

GEO

LEO

0.0022

LEO

2500

GEO

2000

GEO #N/A

1500

0

0.00

GEO

LEO

GEO

LEO

LEO

1000

0.25

2500

#N/A

500

Average failure event per time (FIT)

SAD


0.05

1000

10:20

LEO

426

0.0304

0.0112

240

500

ED Average failure event per time (FIT)

February 17, 2011

0

0.00

GEO

LEO

GEO

LEO

0

Battery Average failure event per time (FIT)

P1: TIX/XYZ P2: ABC JWST062-App02 JWST062-Saleh Printer Name: Yet to Come

189


190

February 17, 2011

10:20


APPENDIX B

Table B.3 Statistical significance of difference in EPS failure behavior in LEO and GEO. p-value Tested samples

Log-rank test

Wilcoxon test

EPS Battery ED SAO

2.29 × 10−12 NA 0.0591 6.42 × 10−11

1.02 × 10−11 0.0408 0.1168 4.44 × 10−12

Table B.4 EPS severity breakdown of anomaly and failure events. EPS events

LEO

GEO

Minor Major Fatal

3 4 10

27 91 8

Total

17

126

EPS failure events in LEO and GEO. Recall that Section B.2 defined three classes of failure events, as minor, major, and fatal. Table B.4 gives the number of EPS failure events by severity category, in both LEO and GEO. The results are also provided in a visually more compact form in Figure B.12. Figure B.12 reads as follows. For example, 17.65% of all EPS failure events in LEO are minor anomalies, 23.53% are major anomalies, and 58.82% are fatal events. minor

LEO

GEO

0%

17.65%

21.43%

10%

major

fatal

23.53%

2

58.82%

72.22%

20% 30% 40% 50% 60% 70% 80% Percentage of anomalies and failures

6.35%

90% 100%

17

1

0

126

20

40

60

80

100 120 140

Number of events

Figure B.12 Percentage breakdown of EPS failure events by severity.


February 17, 2011

10:20


APPENDIX B

191

The important result in Figure B.12 is that fatal EPS events are dominant in LEO. In other words, although EPS failure events in LEO are much less frequent than their counterparts in GEO, when they do occur, they are likely to be catastrophic. Similarly, major EPS anomalies are dominant in GEO. In other words, when an EPS failure event occurs in GEO, it is very likely to be a major anomaly (72.22%). As a consequence, it can be casually said that the EPS fails less frequently but harder (with fatal consequences to the spacecraft) in LEO than in GEO. Figure B.13 provides the percentage breakdown of failure of events by severity for each of the four subsystems that constitute the EPS. The observation made previously at the macro level, that is, the EPS fails less frequently but harder (with fatal consequences to the spacecraft) in LEO than in GEO, also applies to the Battery, ED, and SAO subsystems. Consider the Battery for example. Figure B.13 shows that the majority of its failure events in LEO were fatal to the spacecraft (60%), whereas no Battery fatal

Battery minor

LEO

20.00%

GEO

20.00%

0%

10%

major

20.00%

fatal

2 5

60.00%

1

80.00%


90% 100%

0

15

20

40

60

80

100 120 140

Number of events

ED minor

20.00%

LEO

0%

10%

fatal

2 5

80.00%

38.89%

GEO

major

44.44%


16.67%

90% 100%

1

0

18

20

40

60

80

100 120 140

Number of events

Figure B.13 Percentage breakdown of failure events by severity for each of the EPS elements.


192

February 17, 2011

10:20


APPENDIX B SAD minor

major

fatal

1

100.00%

LEO

33.33%

GEO

0%

10%

50.00%

6

16.67%


90% 100%

0

20

40

60

80

100 120 140

Number of events

SAO minor

33.33%

LEO

17.24%

GEO

0%

10%

major

fatal

16.67%

6

50.00%

78.16%


87

4.60%

90% 100%

0

20

40

60

80

100 120 140

Number of events

Figure B.13 (Continued) event was recorded in GEO in the sample. It is important to note, however, that while the Battery accounts for a small share of EPS failure events in GEO (12% as seen in Figure B.10), most of these events are major anomalies (80%). Similar observations can be made regarding the ED, except that some fatal events in GEO are recorded (16.7%) along with many minor anomalies (33.9%), which was not the case on both counts for the Battery. The data set for the SAD is exceedingly small and does not allow meaningful comparisons of the severity of failure events. The SAO was seen previously as: (1) being the main driver of EPS failure events in GEO; and (2) having the starkest difference in its failure behavior (extent) between LEO and GEO. Figure B.13 adds to these features of the SAO the fact that the severity of its failure events is also distinct in LEO and GEO. While rare, most of the SAO failure events in LEO are fatal (50%), whereas a small fraction of them is fatal in GEO (4.6%). It is important to note, however, that while few events associated with the SAO are fatal in GEO, most of these events constitute major anomalies (78.2%). This


February 17, 2011

10:20


APPENDIX B

193

observation, compounded with the fact that the GEO SAO is the most prolific in terms of failure events for the EPS, makes this subsystem a prime target for improvements, either through better design, testing, and parts selection, or redundancy.

B.6

Conclusion

This appendix provided first a review of the reliability and multi-state failure analysis results of the spacecraft EPS and its constitutive elements. It then proceeded to specialize the EPS anomaly and failure data by orbit (LEO and GEO), following which a comparative analysis of EPS failure behavior in LEO and GEO was conducted. The results proved that, from an actuarial perspective, the EPS fails significantly differently in LEO and GEO, in terms of both extent and severity of failure events. Roughly, an order of magnitude more EPS failure events were found to occur in GEO than in LEO, and they occur more frequently, driven principally by the SAO. This is likely the result of the differences in the space environment between LEO and GEO noted in Section B.3. In addition, it was found that fatal EPS events are dominant in LEO, and that although EPS failure events in LEO are much less frequent than their counterparts in GEO, when they do occur, they are likely to be catastrophic for the whole spacecraft. This may be related to the differences in operational constraints and power cycling in LEO and GEO. A casual summary of the findings in this appendix can be stated as follows: the EPS fails less frequently but harder (with fatal consequences to the spacecraft) in LEO than in GEO. Finally, it is worth pointing out that the available descriptions in the database of failure events are incomplete, and do not allow the analysis of specific EPS failure mechanisms and their distributions. For example, battery failures are described as due to “recharging mechanisms,” or “cell losses,” and SAO failures due to “string losses,” or “short circuits.” Whether different physics of failure or failure mechanisms affects the EPS in LEO and GEO is left as a fruitful avenue for future research.

P1: TIX/XYZ JWST062-REF


February 11, 2011

0:27


References Ajmone Marsan, M. (1989) Stochastic Petri nets: an elementary introduction, in Advances in Petri Nets, Springer, Berlin, pp. 1–29. Ansell, J. I. and Phillips, M. J. (1994) Practical Methods for Reliability Data Analysis, Clarendon Press, Oxford. ANSI/AIAA Guide for Estimating and Budgeting Weight and Power Contingencies for Spacecraft Systems, G-020-1992. Apostol, T. M. (1969) Calculus, Volume II, 2nd edn, John Wiley & Sons, Inc., New York. ARINC Research Corporation (1962) Final Report, Satellite Reliability Spectrum. Report No. 173-5-280, January 30. Atkinson, D. B., Blatt, P., Mahood, L., and Voyls, D. W. (1969) Design of fighter aircraft for combat survivability. Society of Automotive Engineers, National Aeronautic and Space Engineering and Manufacturing Meeting, Los Angeles, Paper 690706. Baker, J. C. and Baker, G. A. (1980) Impact of the space environment on spacecraft lifetime. Journal of Spacecraft and Rockets, 17 (5), 479–480. Ball, R. E. and Atkinson, D. B. (1995) A history of the survivability design of military aircraft. AIAA/ASME/ASCE/AHS/ASC, 36th Structures, Structural Dynamics and Material Conference, New Orleans, Paper AIAA-1995-1421. Barth, J. L. (2005) Prevention of spacecraft anomalies – the role of space climate and space weather models, in Effects of Space Weather on Technology Infrastructure (ed. I. A. Daglis), Kluwer Academic, Dordrecht, pp. 125–128. Bean, E. E. and Bloomquist, C. E. (1968) Reliability data from in-flight spacecraft. 15th Annual Symposium on Reliability, Boston, MA, January 16–18, pp. 271–279. Bearden, D. A. (2003) A complexity-based risk assessment of low-cost planetary missions: when is a mission too fast and too cheap? Acta Astronautica, 52, 371–379. Bedingfield, K. L., Leach, R. D., and Alexander, M. B. (1996) Spacecraft System Failures and Anomalies Attributed to the Natural Space Environment. NASA Reference Publication 1390. Blake, J. B. (1999) Galactic cosmic rays, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA, and Springer, New York, pp. 218–221. Brandhorst, H. W. and Rodiek, J. A. (2008) Space solar array reliability: a study and recommendations. Acta Astronautica, 63 (11–12), 1233–1238. Spacecraft Reliability and Multi-State Failures: A Statistical Approach, First Edition. Joseph Homer Saleh and Jean-François Castet. © 2011 John Wiley & Sons, Ltd. Published 2011 by John Wiley & Sons, Ltd. ISBN: 978-0-470-68791-8



196

February 11, 2011

0:27


REFERENCES

Brown, O. and Eremenko, P. (2006a) Fractionated space architectures: a vision for responsive space. Proceedings of the 4th Responsive Space Conference, Los Angeles, April 24–27, 2006, Paper RS4-2006-1002. Brown, O. and Eremenko, P. (2006b) The value proposition for fractionated space architectures. Proceedings of the AIAA Space 2006 Conference, San Jose, CA, September 19-21, 2006, Paper AIAA-2006-7506. Caralli, R. A. (2006) Sustaining Operational Resiliency: A Process Improvement Approach to Security Management. Carnegie-Mellon University Software Engineering Institute Technical Note CMU/SEI-2006-TN-009. Cassandras, C. G. and Lafortune, S. (2007) Introduction to Discrete Event Systems, 2nd edn, Springer, New York. Castet, J.-F. and Saleh, J. H. (2009a) Satellite reliability: statistical data analysis and modeling. Journal of Spacecraft and Rockets, 46 (5), 1065–1076. Castet, J.-F. and Saleh, J. H. (2009b) Satellite and satellite subsystems reliability: statistical data analysis and modeling. Reliability Engineering and System Safety, 94 (11), 1718–1728. Castet, J.-F. and Saleh, J. H. (2009c) Geosynchronous communication satellite reliability: statistical data analysis and modeling. 27th IET and AIAA International Communications Satellite Systems Conference, June 1-4, Edinburgh. Castet, J.-F. and Saleh, J. H. (2010a) Single versus mixture Weibull distributions for nonparametric satellite reliability. Reliability Engineering and System Safety, 95 (3), 1065–1076. Castet, J.-F. and Saleh, J. H. (2010b) Beyond reliability, multi-state failure analysis of satellite subsystems: a statistical approach. Reliability Engineering and System Safety, 95 (4), 311–322. Castet, J.-F. and Saleh, J. H. (2011) On the concepts of survivability, with application to spacecraft and space-based networks: characterization, stochastic modeling, and analysis. Reliability Engineering and System Safety, in press. Cho, M. (2005) Failure mechanisms and protection methods of spacecraft power system. Proceedings of 2005 International Symposium on Electrical Insulating Materials, Kitakyushu, June 5–9. Coleridge, S. T. (1983) Biographia Literaria, in The Collected Works of Samuel Taylor Coleridge (eds J. Engell and W. J. Bate), Princeton University Press, Princeton, NJ. Coppola, A. (1984) Reliability engineering of electronic equipment: a historical perspective. IEEE Transactions on Reliability, R-33 (1), 29–35. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39 (1), 1–38. Denson, W. (1998) The history of reliability prediction. IEEE Transactions on Reliability, 47 (3-SP), 321–328. DoD Regulation 5000.2-R (1999) Mandatory Procedures for Major Defense Acquisition Programs (MDAPs) and Major Automated Information System (MAIS) Acquisition Programs, May 11. Dubos, G. F., Castet, J.-F., and Saleh, J. H. (2010) Statistical reliability analysis of satellites by mass category: does spacecraft size matter? Acta Astronautica, 67 (5–6), 584–595. Dubos, G. F. and Saleh, J. H. (2011) Comparative cost and utility analysis of monolith and fractionated spacecraft using failure and replacement Markov models. Acta Astronautica, 68 (1–2), 172–184.



February 11, 2011

0:27


REFERENCES

197

Duncan, A. J. (1974) Quality Control and Industrial Statistics, 4th edn, Richard D. Irwin, Homewood, IL. Ellison, B., Fisher, D. A., Linger, R. C., Lipson, H. F., Longstaff, T., and Mead, N. R. (1999) Survivable Network Systems: An Emerging Discipline. Carnegie-Mellon Software Engineering Institute Technical Report CMU/SEI-97-TR-013, May (1997 revised version). Ferguson, D. C., Snyder, D. B., Vayner, B. V. and Galofaro, J. T. (1999) Array arcing in orbit – from LEO to GEO. 37th Aerospace Sciences Meeting and Exhibit, Reno, NV, January 11–14, Paper AIAA-1999-0218. Fleeter, R. (1999) Design of low-cost spacecraft, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA and Springer, New York, pp. 853–863. Fortescue, P. W., Stark, J., and Swinerd, G. (2003) Spacecraft Systems Engineering, 3rd edn, John Wiley & Sons, Inc., Hoboken, NJ, p. 589. Freeman, H. A. (1936) Statistical methods for quality control. Mechanical Engineering, p. 261. Frezet, M., Granger, J. P., Daly, E. J. and Hamelin, J. (1989) Assessment of electrostatic charging of satellites in the geostationary environment. ESA Journal, 13 (2), 89–116. Gindorf, T. E., Miles, R. F. Jr., and Murphy, G. B. (1994) Space-hardware design for long life with high reliability. Proceedings of the Annual Reliability and Maintainability Symposium, Anaheim, CA, January 24–27, pp. 338–341. Gorney, D. J. (1999) Solar particle events, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA and Springer, New York, pp. 217–218. Haas, P. (2002) Stochastic Petri Nets: Modelling, Stability, Simulation, Springer, New York. Haga, R. A. and Saleh, J. H. ( 2011) Epidemiology of satellite anomalies and failures: a subsystem-centric approach. IEEE Aerospace Conference, Big Sky, MT, March 5–12. Hecht, H. (1999) Reliability for mission planning, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA and Springer, New York, pp. 777–778. Hecht, M. and Fiorentino, E. (1987) Reliability assessment of spacecraft electronics. Annual Reliability and Maintainability Symposium, Philadelphia, PA, January 27–29, pp. 341–346. Hecht, H. and Hecht, M. (1985) Reliability Prediction for Spacecraft. Rome Air Development Center Technical Report, RADC-TR-85-229. Hiriart, T., Castet, J.-F., Lafleur, J. M., and Saleh, J. H. (2009) Comparative reliability of GEO, LEO, and MEO satellites. Proceedings of the 60th International Astronautical Congress, Daejeon, October 12–16. Hiriart, T. J. M. and Saleh, J. H. (2010) Observation on the evolution of satellite launch volume and cyclicality in the space industry. Space Policy, 26 (1), 53–60. Hoeber, C. F., Roberston, E. A., Katz, I., Davis, V. A., and Snyder, D. B. (1998) Solar array augmented electrostatic discharge in GEO. 17th AIAA International Communications Satellite Systems Conference and Exhibit, Yokohama, February 23–27, Paper AIAA-19981401. IEEE Std 1012-2004 (2004) IEEE Standard for Software Verification and Validation. IEEE, Piscataway, NJ.



198

February 11, 2011

0:27


REFERENCES

Jiang, X., Wang, Z., Sun, H., Chen, X., Zhao, T., Yu, G., and Zhou, C. (2009) Reliability analysis and utilization of PEMs in space application. Acta Astronautica, 65, 1500–1505. Kalbfleisch, J. D. and Prentice, R. L. (1980) The Statistical Analysis of Failure Data, John Wiley & Sons, Inc., New York. Kang, H., Butler, C., Yang, Q., and Chen, J. (1998) A new survivability measure for military communication networks. IEEE Military Communications Conference, Boston, MA, October 18–21, Vol. 1, pp. 71–75. Kaplan, E. L. and Meier, P. (1958) Nonparametric estimation from incomplete observations. Journal of American Statistical Estimation, 53 (282), 457–481. Katz, I., Davis, V. A., and Snyder, D. B. (1998) Mechanisms for spacecraft charging initiated destruction of solar arrays in GEO. 36th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, January 12–15, Paper AIAA-1998-1002. Kim, S., Castet J.-F., and Saleh, J. H. (2011) Satellite electrical power subsystem: statistical analysis of on-orbit anomalies and failures. 2011 IEEE Aerospace Conference. Big Sky, MT, March 5–12. Knight, J. C., Strunk, E. A., and Sullivan, K. J. (2003) Towards a rigorous definition of information system survivability. Proceedings of DARPA Information Survivability Conference and Exposition, Washington, DC, April 22–24, Vol. 1, pp. 78–89. Koons, H. C. (1999) Plasmas, the magnetic field, and spacecraft charging, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA and Springer, New York, pp. 212–214. Krasich, M. (1995) Reliability Prediction Using Flight Experience: Weibull Adjusted Probability of Survival Method. NASA TR 20060041898. Kvam, P. and Vidakovic, B. (2007) Nonparametric Statistics with Applications to Science and Engineering, Wiley-Interscience. New York. Lafleur, J. M. and Saleh, J. H. (2009) System-level feasibility assessment of microwave power beaming of small satellites. Journal of Propulsion and Power, 25 (4), 976–983. Landis, G. A., Bailey, S. G., and Tischler, R. (2006) Causes of power-related satellite failures. IEEE 4th World Conference on Photovoltaic Energy Conversion, Waikaloa, HI, May 8–12. Lawless, J. F. (2003) Statistical Models and Methods for Lifetime Data, 2nd edn, John Wiley & Sons, Inc., New York. Leventhal, A., Bloomquist, C. E., and Joseph, J. A. (1969) Spacecraft failure rates – where are we? IEEE Proceedings of the Annual Symposium on Reliability, 2 (1), 444–452. Lisnianski, A. and Levitin, G. (2003) Multi-State System Reliability: Assessment, Optimization and Applications, World Scientific, Singapore. Maier, P., Smith, M. R., Keyssar, A., and Kevles, D. J. (2003) Inventing America: A History of the United States, W. W. Norton, New York. McDermott, J. K. (1999) Power, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA and Springer, New York, p. 420. McLachlan, G. J. and Krishnan, T. (2008) The EM Algorithm and Extensions, 2nd edn, WileyInterscience, New York. Meeker, W. O. and Escobar, L. A. (1998) Statistical Methods for Reliability Data, John Wiley & Sons, Inc., New York. MIL-HDBK-217 (1965–1995) Military Handbook, Reliability Prediction of Electronic Equipment.



February 11, 2011

0:27


REFERENCES

199

MIL-HDBK-336-1 (1982) Military Handbook, Survivability, Aircraft, Nonnuclear, General Criteria, Vol. 1. October 25. MIL-HDBK-2069 (1997) Military Handbook, Aircraft Survivability, April 10. MIL-STD-2069 (1981) Military Standard Requirements for Aircraft Nonnuclear Survivability Program, August 24. Norris, H. P. and Timmins, A. R. (1976) Failure rate analysis of Goddard Space Flight Center spacecraft performance during orbital life. Annual Reliability and Maintainability Symposium, Las Vegas, NV, January 20–22, pp. 120–125. Panel on Small Spacecraft Technology, National Research Council (1994) Technology for Small Spacecraft, National Academy Press, Washington, DC. Peterson, J. L. (1977) Petri nets. Computing Surveys, 9 (3), 223–252. Peterson, J. L. (1981) Petri Net Theory and the Modeling of Systems, Prentice Hall, Englewood Cliffs, NJ. Rausand, M. and Høyland, A. (2004) System Reliability Theory: Models, Statistical Methods, and Applications, 2nd edn, Wiley-Interscience, New York, pp. 465–524. Raymond Knight, C. (1991) Four decades of reliability progress. Proceedings of the IEEE Annual Reliability and Maintainability Symposium, Orlando, FL, January 29–31, pp. 156–160. Robertson, B. and Stoneking, E. (2003) Satellite GN&C anomaly trends. AAS Guidance and Control Conference, Breckenridge, CO, February 5–9. Robinson, P. A. (1989) Spacecraft Environmental Anomalies Handbook, GL-TR-89-0222, Air Force Geophysics Laboratory, Hanscom Air Force Base, MA. Rodiek, J. A., Brandhorst, H. W., and O’Neill, M. J. (2008) Stretched lens solar array: the best choice for harsh orbits. 6th International Energy Conversion Engineering Conference, Cleveland, OH, July 28–30, Paper AIAA-2008-5755. Saleh, J. H., Lamassoure, E., Hastings, D. E., and Newman, D. J. (2003) Flexibility and the value of on-orbit servicing: a new customer-centric perspective. Journal of Spacecraft and Rockets, 40 (1), 279–291. Saleh, J. H., and Marais, K. (2006) Highlights from the early (and pre-)history of reliability engineering. Reliability Engineering and System Safety, 91 (2), 249–256. Sarsfield, L. P. (1998) The Cosmos on a Shoestring – Small Spacecraft for Space and Earth Science, RAND, Santa Monica, CA, MR-864-OSTP. Schulz, M. and Vampola, A. L. (1999) Trapped radiation, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA and Springer, New York, pp. 214–216. Snyder, D. (1982) Environmentally induced discharges in a solar array. IEEE Transactions on Nuclear Science, 29, 1607–1609. SpaceTrak, Ascend Worldwide [online database], http://www.ascendworldwide.com/whatwe-do/ascend-data/space-data/space-trak.html (retrieved May 26, 2010). Sperber, R. (1990) Analysis of the public record of spacecraft anomalies. 13th AIAA International Communication Satellite Systems Conference and Exhibit (ICSSC), Los Angeles, March 11–15, pp. 42–51. Sperber, R. (1994) Better with age and experience – observed satellite in-orbit anomaly rates. 15th AIAA International Communication Satellite Systems Conference and Exhibit (ICSSC), San Diego, CA, February 28–March 3, pp. 1162–1167. Sperber, R. (2002) Hazardous subsystems. SatMax 2002: Satellite Performance Workshop, Arlington, VA.



200

February 11, 2011

0:27


REFERENCES

Surampudi, R., Bugga, R., Smart, M. C., and Narayanan, S. R. (2006) Overview of Energy Storage Technologies for Space Applications, Jet Propulsion Laboratory, Pasadena, CA. Tafazoli, M. (2009) A study of on-orbit spacecraft failures. Acta Astronautica, 64 (2–3), 195–205. Titterington, D. M., Smith, A. F. M., and Makov, U. E. (1985) Statistical Analysis of Finite Mixture Distributions, John Wiley & Sons, Inc., New York. Timmins, A. R. (1974) A Study of the First-Month Space Malfunctions, NASA Technical Report TN-D-7750. Timmins, A. R. (1975) A Study of the Total Space Life Performance of GSFC Spacecraft, NASA Technical Report TN-D-8017. Timmins, A. R. and Heuser, R. E. (1971) A Study of First-Day Space Malfunctions, NASA Technical Report TN-D-6474. Vayner, B. V., Ferguson, D. C., and Galofaro, J. T. (2007) Comparative analysis of arcing in LEO and GEO simulated environments. 45th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, January 8–11, Paper AIAA-2007-0093. Visentine, J. T. (ed.) (1988) Atomic Oxygen Effects Measurements for Shuttle Mission STS-8 and 41-G, NASA TM-100459. Volovoi, V. (2004) Modeling of system reliability Petri nets with aging tokens. Reliability Engineering and System Safety, 84 (2), 149–161. Volovoi, V. (2006) Stochastic Petri nets modeling SPN@. Reliability and Maintainability Symposium (RAMS), Newport Beach, CA, January 26–29, Paper 2006RM-166. Walker, B., Holling, C. S., Carpenter, S. R., and Kinzig, A. (2004) Resilience, adaptability and transformability in social–ecological systems. Ecology and Society, 9 (2), 5 [online: http://www.ecologyandsociety.org/vol9/iss2/art5/]. Walterscheid, R. L. (1999) The upper atmosphere, in Space Mission Analysis and Design, 3rd edn (eds J. R. Wertz and W. J. Larson), Microcosm Press, Hawthorne, CA and Springer, New York, pp. 207–212. Wertz, J. R. and Larson, W. J. (eds) (1999) Space Mission Analysis and Design, 3rd edn, Microcosm Press, Hawthorne, CA and Springer, New York. Westmark, V. R. (2004) A definition for information system survivability. Proceedings of the 37th Annual Hawaii International Conference on System Sciences, Big Island, HI, January 5–8. Williamson, M. (2006) Spacecraft Technology: The Early Years, Institute of Electrical Engineers, Stevenage, p. 306. Xie, L., Smith, P., Banfield, M., Leopold, H., Sterbenz, J. P. G., and Hutchison, D. (2005) Towards resilient networks using programmable networking technologies. Seventh Annual International Working Conference on Active and Programmable Networks (IWAN 2005), Sophia Antipolis, November.

P1: TIX/OSW JWST062-IND


February 17, 2011

9:50


Index Battery subsystem, anomaly severity, 86–90, 191 confidence interval, 73, 183 contribution to spacecraft failure, 79–81 cycling, 66, 182 depth of discharge, 66, 182 failure by orbit, 188 failure modes, 80 health scorecard, 173 maximum likelihood estimates, 76 multi-state failures, 110, 121, 184 nonparametric reliability, 73, 183 time to anomaly and failure, 93 transition probabilities, 110 Weibull model, 76, 110 Box plot, 32 Censoring, classes, 11 definition, 11 Kaplan-Meier estimator, 15 left, 11 multi-state failure analysis, 101–3 random, 11 reliability analysis, 11–12, 15–16 right, 11 type 4, 11 Communication subsystem, anomaly severity, 86–90 health scorecard, 170

multi-state failures, 109, 119 time to anomaly and failure, 96 transition probabilities, 109 Weibull model, 109 Confidence interval, analysis, 17–20 dispersion, 19 Greenwood formula, 18 multi-state failure analysis, 103 normal assumption, 18 spacecraft reliability, 18 Control processor subsystem, anomaly severity, 86–90 confidence interval, 73 contribution to spacecraft failure, 79–81 health scorecard, 170 maximum likelihood estimates, 76 multi-state failures, 109, 119 nonparametric reliability, 73 time to anomaly and failure, 96 transition probabilities, 109 Weibull model, 76, 109 Database, censoring, 10–11 completeness, 10–11 data collection, 11 description, 10 Design lifetime, 47



202


February 17, 2011

9:50


INDEX

Electrical distribution subsystem, anomaly severity, 86–90, 191 confidence interval, 74, 183 contribution to spacecraft failure, 79–81 failure by orbit, 188 health scorecard, 174 maximum likelihood estimates, 76 multi-state failures, 110, 121, 184 nonparametric reliability, 74, 183 time to anomaly and failure, 93 transition probabilities, 110 Weibull model, 76, 110 Electrical power subsystem (EPS), see also Battery subsystem, Electrical distribution subsystem and Solar array subsystem anomaly severity, 86–90, 190 confidence interval, 73–4, 183 contribution to spacecraft failure, 79–81 cycling, 66, 182 failure by orbit, 185–6 failure description, 193 health scorecard, 173–6 maximum likelihood estimates, 76 multi-state failures, 110, 121–2, 184 nonparametric reliability, 73–4, 183 operational constraints, 182 time to anomaly and failure, 93–4 transition probabilities, 110 Weibull model, 76, 110 Expectation-Maximization algorithm, 35 Extreme value distribution, log-likelihood function, 29 maximum likelihood estimation, 29 probability density function, 29 reliability function, 29 Failure rate, definition, 54 GEO communication spacecraft, 157 spacecraft, 56

reliability determination, 54 roller-coaster, 157 Fractionation, 131 Graphical estimation, see Probability plot Goodness of fit, 32–3, 37–42, 49–51, 52–5, 62, 77, 111–13, 158–9 Greenwood formula, 18 Gyro subsystem, anomaly severity, 86–90 confidence interval, 73 contribution to spacecraft failure, 79–81 health scorecard, 168 maximum likelihood estimates, 76 multi-state failures, 105, 109, 111–15 nonparametric reliability, 73 stochastic Petri net model, 118 time to anomaly and failure, 91 transition probabilities, 105, 109, 111 Weibull model, 76, 109, 111 Hypothesis test, log-rank, 60, 188 Wilcoxon, 60, 188 Infant anomaly, 92–94, 167 Infant degradation, 115, see also Infant anomaly and Infant mortality Infant mortality, 6, 17, 24, 31, 37, 43, 47, 49, 52, 56, 60, 63, 65, 75–6, 80, 92–4, 115, 156–9, 167 causes, 6 subsystem, 75–6, 80 testing, 63 Kaplan-Meier estimator, asymptotic normal property, 18 derivation, 15–16 maximum likelihood estimator, 18 multi-state failures, 102–3 standard deviation, 18 ties, 16 variance, 18



February 17, 2011

9:50


INDEX

Kaplan-Meier plot, definition, 16 multi-state failures, 104–7 spacecraft reliability, 17 Likelihood function, see also Maximum likelihood estimation censoring, 28 definition, 28 log-likelihood function, 29, see also Log-likelihood function mixture distribution, 35 Log-likelihood function,see also Maximum likelihood estimation definition, 29 extreme value distribution, 29 gradient, 29, 36 Hessian, 29 mixture distribution, 35 Weibull distribution, 29 Lognormal distribution parameters, 33 probability density function, 33 Maximum likelihood estimation, derivation for Weibull distribution, 28–9 description, 27–8 general derivation, 28 likelihood function, 28, see also Likelihood function log-likelihood function, 28, see also Log-likelihood function maximum likelihood estimate, 28 maximum likelihood estimator, see Maximum likelihood estimator optimization, 28, 34–6 Maximum likelihood estimator, asymptotic normal property, 18 Kaplan-Meier estimator, 18 Mixture distribution, definition, 33–4 likelihood function, 35 log-likelihood function, 35

203

maximum likelihood estimation, 34–6 parameter estimation methods, 34 probability density function, 35 reliability function, 35 Weibull distribution, 34 Multi-state failure analysis, censoring, 101–2 confidence interval, 103 dimension damnation, 104 Kaplan-Meier estimator, 102 maximum likelihood estimation, 108 parametric modeling, 108 states, 97, 100 stochastic Petri net, 110 ties, 103 transition diagram, 100 transition probabilities, 109–10 verification and validation, 109 versus reliability analysis, 113 Weibull modeling, 108 Nonparametric estimation, advantages, 23 censored data, 11–12 complete data, 13–14 Kaplan-Meier estimator, see Kaplan-Meier estimator limitations, 23 Normal probability plot, 41–2 Orbit, eclipse, 66, 182 GEO, 57 LEO, 57 Parametric modeling, maximum likelihood estimation, see Maximum likelihood estimation methods, 23 probability plot, see Probability plot Payload subsystem, anomaly severity, 86–90 confidence interval, 73


204


February 17, 2011

9:50


INDEX

Payload subsystem (Continued ) contribution to spacecraft failure, 79–81 health scorecard, 172 maximum likelihood estimates, 76 multi-state failures, 109, 120 nonparametric reliability, 73 time to anomaly and failure, 95 transition probabilities, 109 Weibull model, 76, 109 Petri net, definition, 129 directed arc, 129 dynamic properties, 129 enabler arc, 130 inhibitor arc, 130 marking, 129 place, 129 static properties, 129 stochastic, see stochastic Petri net token, 129 transition, 129 Probability plot, advantages, 26 definition, 25 limitations, 27 normal probability plot, 41–2 Weibull plot, 25 Product-limit estimator, see Kaplan-Meier estimator Reliability, binary concept, 83, 97 complete data, 13–14 definition, 1, 4 empirical function, 13–14 extreme value distribution, 29 failure rate determination, 54 GEO communication spacecraft, 152 iid, 13 Kaplan-Meier estimator, 14–16 mixture distribution, 34 nonparametric, 9 parametric, 23

probability plot, 25 stochastic Petri net, 129, 139 test bed, 149 ties, 14 Weibull distribution, 24 Reliability engineering, 1–4 Resiliency, definition, 126 versus survivability, 127 Solar array, anomaly severity, 86–90, 192 confidence interval, 74, 183 contribution to spacecraft failure, 79–81 corrosion, 181 cycling, 66, 182 failure by orbit, 188 health scorecard, 162, 175–6 maximum likelihood estimates, 76 multi-state failures, 110, 122, 185 nonparametric reliability, 74, 183 time to anomaly and failure, 94 transition probabilities, 110, Weibull model, 76, 110 Space environment, arcing, 65, 181 corrosion, 65, 181 drag, 65, 181 electromagnetic interference, 65, 181 electrostatic discharge, 65, 181 galactic cosmic ray, 66, 181 plasma, 65, 66, 181 radiation belt, 66, 181 single-event, 66, 182 solar particle event, 66, 181 upper atmosphere, 65, 181 Space-based network fractionation, 131 survivability, 142, 146 Spacecraft, budget, 5, 63 complexity, 44, 65



February 17, 2011

9:50


INDEX

cost, 5 COTS, 63 design lifetime, 47 failure causes, 7 GEO communication, 151 GEO platforms, 160 launch rate, 5 mass, 44, 46 mission, 5 multi-state failures, 97 packaging density, 64 redundancy, 64 reliability, 5 shielding, 64 state of health, 80, 149 stochastic Petri net, 136 subsystem, 72 subsystem failures, 7, 71, 83, 97 survivability, 128, 142 testing, 63 Statistical significance, hypothesis test, 60, 188 level, 60 p-value, 60 Stochastic Petri net, see also Petri net definition, 129 for survivability, 131 Monte Carlo simulation, 131, 140–1 verification and validation, 138–40 versus Markov Chain, 131 Structure subsystem, anomaly severity, 86–90 confidence interval, 73 contribution to spacecraft failure, 79–81 health scorecard, 171 maximum likelihood estimates, 76 multi-state failures, 109, 120 nonparametric reliability, 73, time to anomaly and failure, 95 transition probabilities, 109 Weibull model, 76, 109

205

Subsystem, see also individual subsystems anomaly severity, 86–90 confidence interval, 73–4 contribution to spacecraft failure, 79–81 failure concentration ratio, 163 health scorecard, 168–77 infant anomaly, see Infant anomaly infant mortality, see Infant mortality multi-state failures, 97 nomenclature for, 72 nonparametric reliability, 73–4 time to anomaly and failure, 90–6 Weibull model, 76, 109–10 wear-out, see Wear-out Survivability, definition, 125–6 framework, 128 Markov Chain, 131 modeling tools, 128 network, 125, 128, 142–3 recoverability, 125 space-based network, 128, 142–3 spacecraft, 128, 142–3 stochastic Petri net, 131 subsystem states, 135 susceptibility, 125 system states, 133, 135 versus resiliency, 127 vulnerability, 125 Telemetry, tracking, and command subsystem, anomaly severity, 86–90 confidence interval, 74 contribution to spacecraft failure, 79–81 health scorecard, 177 maximum likelihood estimates, 76 multi-state failures, 107, 110, 117–18 nonparametric reliability, 74 time to anomaly and failure, 95


206


February 17, 2011

9:50


INDEX

Telemetry, tracking, and command subsystem (Continued ) transition probabilities, 107, 110 Weibull model, 76, 110 Thermal subsystem, anomaly severity, 86–90 confidence interval, 73 contribution to spacecraft failure, 79–81 cycling, 66, 182 health scorecard, 171 maximum likelihood estimates, 76 multi-state failures, 109, 120 nonparametric reliability, 73 time to anomaly and failure, 95 transition probabilities, 109 Weibull model, 76, 109 Thruster subsystem, anomaly severity, 86–90 confidence interval, 73 contribution to spacecraft failure, 79–81 health scorecard, 169 maximum likelihood estimates, 76 multi-state failures, 104, 106, 109, 111–12, 115–17 nonparametric reliability, 73

time to anomaly and failure, 91 transition probabilities, 104, 106, 109, 111 Weibull model, 76, 109, 111 Wear-out, 6–7, 24, 37, 48, 51–2, 56, 60, 64, 92–3, 156–9, 167 Weibull distribution, equivalent extreme value distribution, 29 failure rate, 24 flexibility, 24 graphical estimation, 25 maximum likelihood estimation, 28–9, 34–6 mixture distribution, 34 parameters, 24 probability density function, 24 probability plot, 25–6, see also Weibull plot reliability function, 24 Weibull plot, definition, 25 GEO communication spacecraft, 156, 160 spacecraft reliability, 25 thruster subsystem, 75

Spacecraft Reliability and Multi-State Failures: A Statistical Approach

Process Optimization: A Statistical Approach

Process optimization: A statistical approach

Process Optimization: A Statistical Approach

Spacecraft Dynamics and Control: A Practical Engineering Approach

Statistical Methods for Reliability Data

Statistical Methods for Reliability Data

Spacecraft Dynamics and Control: A Practical Engineering Approach

Mathematical and Statistical Methods in Reliability

Nonlinear Signal Processing: A Statistical Approach

Nonlinear Signal Processing: A Statistical Approach

Quantum Geometry. A Statistical Field Theory Approach

Quantum Geometry: A Statistical Field Theory Approach

Nonlinear Signal Processing: A Statistical Approach

Nonlinear Signal Processing: A Statistical Approach

Nonlinear signal processing: a statistical approach

Rocket and Spacecraft Propulsion

Modern Statistical And Mathematical Methods in Reliability (Quality, Reliability and Engineering Statistics)

Multistate and Multinational Estate Planning Volume 2

Spacecraft dynamics

Modern Statistical And Mathematical Methods in Reliability (Quality, Reliability and Engineering Statistics)

Evolution and Structure of the Internet: A Statistical Physics Approach

Statistical methods: The geometric approach

Statistical Inference: An Integrated Approach

STATISTICAL METHODS: THE GEOMETRIC APPROACH

Spacecraft Sensors

Spacecraft Structures

Multistate and Multinational Estate Planning Volume 1

Reliability Criteria in Information Theory and in Statistical Hypothesis Testing

Spacecraft Reliability and Multi-State Failures: A Statistical Approach

Process Optimization: A Statistical Approach

Process optimization: A statistical approach

Process Optimization: A Statistical Approach

Spacecraft Dynamics and Control: A Practical Engineering Approach

Statistical Methods for Reliability Data

Statistical Methods for Reliability Data

Spacecraft Dynamics and Control: A Practical Engineering Approach

Mathematical and Statistical Methods in Reliability

Nonlinear Signal Processing: A Statistical Approach

Nonlinear Signal Processing: A Statistical Approach

Quantum Geometry. A Statistical Field Theory Approach

Quantum Geometry: A Statistical Field Theory Approach

Nonlinear Signal Processing: A Statistical Approach

Nonlinear Signal Processing: A Statistical Approach

Nonlinear signal processing: a statistical approach

Rocket and Spacecraft Propulsion

Modern Statistical And Mathematical Methods in Reliability (Quality, Reliability and Engineering Statistics)

Multistate and Multinational Estate Planning Volume 2

Spacecraft dynamics

Modern Statistical And Mathematical Methods in Reliability (Quality, Reliability and Engineering Statistics)

Evolution and Structure of the Internet: A Statistical Physics Approach

Statistical methods: The geometric approach

Statistical Inference: An Integrated Approach

STATISTICAL METHODS: THE GEOMETRIC APPROACH

Spacecraft Sensors

Spacecraft Structures

Multistate and Multinational Estate Planning Volume 1

Reliability Criteria in Information Theory and in Statistical Hypothesis Testing

Recommend Documents