Nonlinear prediction ladder-filters for higher-order stochastic sequences

Lecture Notes in Control and Information Sciences Edited by M.Thoma

73 III

IIII

IIIIIIIIII

IIIII

J. Zarzycki

Nonlinear Prediction

Ladder-Filters for Higher-Order Stochastic Sequences I

IIIIIII

II

IIIIIIIIIIIIIIIIIIII

IIIIIIIIIIIIIII

Springer-Verlag Berlin Heidelberg New York Tokyo

Series Editor M.Thoma Advisory Board A.V. Balakrishnan • L. D. Davisson • A. G. J. MacFarlane H. Kwakernaak • J. L. Massey • Ya Z. Tsypkin • A. J. Viterbi Author Jan Zarzycki Institute of Telecommunication and Acoustics The Technical University of Wroclaw ul. B. Prusa 5 3 / 5 5 50-317 Wroclaw - Poland

ISBN 3-540-15635-6

Springer-Verlag Berlin Heidelberg New York Tokyo

ISBN 0-38?-15635-6

Springer-Verlag New York Heidelberg Berlin Tokyo

Library of Congress Cataloging in Publication Data Zarzycki, J. (Jan) Nonlinear prediction ladder-filters for higher-order stochastic sequences. (Lecture notes in control and information sciences; 73) Bibliography: p. 1. Stochastic sequences. 2. Prediction theory. 3. Filters (Mathematics) I. Title. I1. Series. QA274.225.Z37 1985 519.2 85-12668 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks. Under c:354 of the German Copyright Law where copies are made for other than private use, a fee is payable to "Verwertungsgesellschaft Wort", Munich. © Springer-Verlag Berlin, Heidelberg 1985 Printed in Germany Offsetprinting: Mercedes-Druck, Berlin Binding: LiJderitz und Bauer, Berlin 2161/3020-543910

PREFACE

In t h i s w o r k w e s h a l l b e c o n c e r n e d with t h e p r o b l e m of n o n l i n e a r

least-squares linear

prediction of h i g h e r - o r d e r stochastic s e q u e n c e s

orihogonal

digital

filters. The

nonlinear

problem

as a generalization of the lineaLr least-squares The cond-order

linear l e a s t - s q u a r e s

will be

prediction problem.

for w h i c h

white noise

when

ven sequence)

or s h a p i n g

filters ( w h o s e

modeling

statistically equivalent to the given s e q u e n c e ,

when

sire p r o c e d u r e s ,

in w h i c h

one

will not h a v e

e a c h time the permitted complexity derlies both the ladder-structures ry, a n d

the theory of orthogonal

the gi-

output is

driven b y white noise)

remarkable

implemented

Hence,

expa~sions

means

plemented

using

(namely CORDICS

of a n

any

orthogonal filter w h o s e

via recur-

the w h o l e

the s a m e

in m o d e r n

inherent numerica/

result of this theory is t h a t

alized b y

The

assures

computed

to r e c o m p u t e

is increased.

(Fourier)

be

of the m o s t importa~nt proper~y of the orthogonal

vat[on of 'energy' w h i c h A

driven b y

established° In practice, the linear or£hogonal filters c a n

One

with s e -

the orthogonal prediction or

innovations linear filters ( p r o d u c i n g

can be

considered

estimation theory is a s s o c i a t e d

stochastic s e q u e n c e s ,

as well a s

using non-

filter

idea u n -

digital filters theo-

in Hilber~

spaces.

digital filter is p r e s e t stability of the filter.

transfer function c a n modular

structure c a n

sophisticated 'building-blocks' with V L S I

be be

reim-

integrated circuits

processors).

linear theory results in the o p t i m u m

approximation of s e c o n d - o r d e r

sequences.

(least-squares)

Therefore,

stochastic

the linear estimation

IV filter b e c o m e s properties

the best possible

are completely c h a r a c t e r i z e d

If the undertyin~ s e q u e n c e may

be

In this w o r k least-squares

we

wish

sequence

the s e c o n d - o r d e r

(whose

statistics).

the linear estimation a c c u r a c y

a norz[inear a p p r o a c h

in order to i m p r o v e

to the p r o b l e m

the a c c u r a c y .

to p r e s e n t efficient algorithms of nonlinear

prediction filters for higher-order stochastic s e q u e n c e s ,

sulting in the o p t i m u m

approximate

re-

nonlinear digital filters of the Volterra-

class, ri~hese nonlinear ladder-filters will generalize the linear fil-

ters, p r e s e r v i n g zations, a m o n g order

by

is non-Gaussio.n,

not satisfactory. In that c ~ e ,

shot~Id b e introduced

Wiener

filter for ~ Gatlssi~n

m o s t of their properties

will mention h e r e

reali-

for higher-

stocha.stic s e q u e n c e s .

only t h o s e p a p e r s

ted to the subject of this work, the p a p e r s

modular

others), a n d yielding better estimation a c c u r a c y

(o.nd non-CTaussi~.n)

We

(orthogonality a n d

which

reffering for m o r e

citied ( a n d the r e f e r e n c e s

are closely c o n n e c -

complete

bibliography to

therein),

ACKNOWLEDGMENTS

I am p a r t i c u l a r l y indebted to Professor Patrick DewZlde of the D e l f t University of Technology for ~is helpful suggestions and h i n ~ introduced in many stim~at~ng and f r u i t f ~

disc~sio~,

~ p e c i a ~ l y d~ring my o n e - y e n s t a y

in Delft, which h ~ undoubtedly inspired t h i s work. I am ~ o

g r a t e f u l to P r o f ~ s o r M~ian S. P i e k ~ s k i of the Technical

Unive~Zty of Wro~aw for his valuable commen~ and d ~ c ~ s i o n s

concerning

t h ~ work. I wish to thank ~ . manuscript.

Zdzislawa Zabska for her c ~ e f u l typing of t h i s

CONTENTS

CHAPTER

i, I N T R O D U C T I O N

CHAPTER

2. N O N L I N E A R A UNIFIED

.......,........................................... ............... PREDICTION APPROACH

I

FILTER PROBLEM: .................................................... 1 3

2.1 H i g h e r - o r d e r stochastic sequences ........................................... 1 3 2.2 N o n l i n e a r lea~st-squares prediction: Algebraic a p p r o a c h ......... 20 2.3 N o n l i n e a r least-squares prediction: Geometric a p p r o a c h ........ 24: 2.3.1 S p a c e of t h e r e g u l a r V o l t e r r a f u n c t i o n a l p o l y n o m i a l s .......................... ............................................ 2 4 2.3.2 S p a c e of g e n e r a l i z e d c o e f f i c i e n t - m a t r i c e s .................. 26 2.3.3 S p a c e of g e n e r a l i z e d z - p o l y n o m i a l s ........................... 2 9 2.3.4 I s o m e t r i e s ........................................................................ 3 2 2.3.5 S t o c h a s t i c n o n l i n e a r e s t i m a t i o n .................................... 3 5 2.3.6 O p t i m u m generalized matrix approximaUon ................. 36 2.3.7 O p t i m u m generalized polynomial a p p r o x i m a t i o n .......... 3 8 CHAPTER

3.

GENERALIZED

NONLINEAR

LA/DDER-F'ILTERS

.............

4O

3.1 I n d e x - s e t s a n d their o r d e r i n g ...................................................... 4 1 3.2 N o n l i n e a r filter algorithm: t i m e - d o m a i n a p p r o a c h ....................... 4 8 3.2.1 'Local' e s t / m a t e s a n d errors..• ......................... ............. 5 1 of s u b s p a c e s ......................................... 5 5 3.2.2 D e c o m p o s i t i o n b a s e s .......................................................... 5 7 3.2.3 O r t h o n o r m a l Cholesky factorizat.[ons ............................ 6 0 3.2.4 G e n e r a l i z e d F o u r i e r s e r i e s e x p a / n s i o n ..................................... 6 1 3.2.5 M - D r e c u r s i o n s ................................................ 6 2 3.2.6 O r d e r - u p d a t e approxim~ion of t h e M - D impulse 3.2.7 O p t i m u m responses•..•••....•.••••.•••••....••.•.•.•...•......•..•.....••••...•.••.•.•.....7 0 3.2.8 E s t i m a t i o n a c c u r a c y ........................................................ 7 1 3.3 N o n l i n e a r filter algorithm: t r a n s f o r m - d o m a i n a p p r o a c h .............. 7 4 3.3.1 'Local' e s t i m a t e s a n d e r r o r s ......................................... 7 6 3.3.2 D e c o m p o s i t i o n of s u b s p a c e s , ON bases and M-D ~'~ourier e x p a n s i o n ................................................. 7 9 3.3.3 0 r d e r - u p d a t e r e c u r s i o n s ................................................ 8 3 3.3.4 Optimum ON approximation of t h e s e t of M - D t r a n s f e r f u n c t i o n s ............................................................ 8 5 3.4 N o n l i n e a r time-variax:t ladder~-filter.............................................. 8 6 CHAPTER

4.1

4. T I M E - I N V A R I A N T LADDER-FILTERS Shift-invariance

AND 'QUASI-LINEAR' ................................................................ 9 1

of inner--produc~......... ........................... ...........

91

6.2 T i m e - i n v a r i a n t n o n l i n e a r ladder-filter ~Igorithm......................... 4.3 ' Q u a s i - l i n e a r ' l a d d e r - f i l t e r s............................................................ 4.4 E x p e r i m e n t a l e x a m p l e.....................................................................

93 98 106

CONCLUDING

REMARKS

REFERENCES

..........................................................................

109

............................................................................................. II0

APPENDIX

i .................................................................................... 116 • , * • t • • •,,• •

APPENDIX

2

.................................................................. ........... ....•........ ..... .

127

i. I N T R O D U C T I O N

(a,B,u)

Let

tract set w h o s e sets of

elements

a, a n d

will u n d e r s t a n d which

~s

denote

U

are

12du




expressed

0

(2.22)

r e n o r m a l i z e d form a s

N N N Z ~ ... ~ aN;il...j m y_jl...y_j m J l = l j2---jl jm=~Jm_l

(2.23a)

where

aN; o =

Md~ql

;

aN;Jl.,,jm

In o r d e r to solve the M - t h

= -

degree

as

N;JI...j m

nonlinear, N - t h o r d e r leo.st-squ~res

21 prediction problem, w e

wish to c h o o s e

the coefficients

such w a y that the m e a n - s q u a r e

error (2.22)

achieved if for

u i (kl,...,ku) ¢ s y m L N _ 1

u=i,...,M

and

aN;Jl...j m

in a

is minimized. This will

be

OIVIR c~N

=

(2.24~)

o

8 aN;kf..ku

Conditions

will imply

(2.24a) M

aN;oho,kl...k u +

Moreover,

E m=l jl=l

N ""

j m=Jm_i

aN;jl...j m h.Jl...Jmkl..*k . u

-

0

(2.2,~b)

get

we

M

5q;ohoo +

N

E

N

N

X E ... E h. Md N m = l jl=l jmmJm_l aN;Jl'"Jm Jl""Jm '° =

G e n e r a l i z i n g t h e notion of 'matrix', we c a n i n t r o d u c e t h e following M-block ( r o w ) ,

m - i n d e x e d coefficient-matrix

{M}AN = [ m A N

whoso

each m-indexed

block-entry

from the m-variate index-set

m%

DmAN

con be considered

D mA N

=

numbers

(2.2~b)

can talk e~bout the 'doma/n' of the m-indexed

that m - v a r i a t e index-set, and

~s a m a p

into the real (or complex)

m%. D m%, -~

Hence, w e

(2.25a)

] m = l .....M

matrix es being

we will have

symmLNI-I

if

m=l

if

m=2,...,M

(2.25c)

22 Consequently,

we write

for

m-2,,,,,M

mAN .. [ °N;Jl""Jm ] and for

m

(Jl .....jm)

e

(2.25d)

i

sym LN_1

m=l

1AN -', [ aN;ill JlC LN Thus,

the M-block,

considered

m-indexed

as a m a p

(2.25e)

coefficient-matrix

{M ~AN

(2.25a)

can be

from the vector of simple d o m a i n s

D{M} %

[DmAN] m~,l,00,,M

=

(2.26a)

into the reat (or complex) numbers

{ M}AN. D{M }AN + ~

see

a/so

Appendix

Using

where

1_yo N

1 mYN_ 1 can

i.

(2.25),(2.26)

{M}YN

=

a n d introducing

M

col [ 1 Y No

is e x p r e s s e d

( m - 2 .....M )

(2.26b)

by

(2.17b)

are given b y

rewrite the error (2.23)

with

(2.17b)

re=l, x = 0 with

in a generalized

MeN; ° = {M }AN. {M}YN

where

•

l

YN-1 ]

"'"

x=l

(2.~T~)

and and

n=N

while

n=N-:i , w e

matrix form a.s follows

(2.27b)

indicates product of generalized matrices ( s e e Appendix 1),

23

Introducing the following matrix [ M~ N

where

Md N

i

(2.28a)

ON_ 1 ] { M}~I

is e x p r e s s e d b y ( 2 . 2 2 ) , and

M-block ( r o w ) ,

1

ON-1

u

block-entries

u

=

1 ON_ I

[Uol_1 ]

are u-indexed

(see also A p p e n d i x

mad equat/ons'

in a generalized

rzycki and Dewilde

denotes the

(2.28b)

u=i,...,M

i i D ON_ l - symuLN_l (2.24)

UN-I

zero-matrix

u-indexed

{M}

whose

{M}

zero-mafrices i), w e

with domains

can rewrite the 'nor-

mafrix form a~s follows (see Za-

(1983a))

t =N where {M}Y N

{ M×M}H N

is e x p r e s s e d b y (2.18)

( 2 . 2 7 a ) , Rewriting

{ M x M }HN

(2.29)

with

{M}YXn r e p l a c e d b y

in a generalized

block-column

form

{MXM}HN.,

[{M}x%

] U-1 .....M

(2,30a)

where for u-2,...,M

{M}xUHN .

[ {M} 1 1 HN;kl"'ku] (kl...,ku)¢ myrau LN_

(2.30b)

and {M}×XHN = [ {M}HN;k 1 ]

0 kI ¢ LN

(2,30C)

with { M}H N;kl...ku - [ h.]l...Jmkl...ku . ] (Jl ..... Jm ) ~ D{M}YN

(2.30d)

24 we

c a n express the 'normal equations' (2.29), for

u=l,...,M

and

for

i

( k 1 ..... ku) ~ s y m u L N _ l , a s follows

(M}^

~'4"

{M) H

{M }AN.( M }HN;o

These

(2.31a)

N;kl..,k u = 0

=

MdN

(2.31b)

are the 'normal equations' associated with the M-th degree nonli-

near, N-th order prediction problem. W e

can observe

that they will reduce

to the 'normal equations' corresponding to the linear problem, if Equations

(2.31)

M~I.

can be recurslvely solved via the nonlinear Le-

vinson algorithm, presented in Zarzycki and Dewilde (1983). q)his algorithm computes

(1983a); Z~vzycki

the coefficients of the optimum nonlinear

approximate prediction ladder-filter, a n d implies the generalized C h o l e s k y factoriza~on of the block, multi-lndexed covariance matrix

{ M×M

}HN ,

generalizing the linear case, considered in meprettere a n d Lie (1980).

2.3 Nonlinear ! e ~ t - s q u a r e s

prediction: Geometric

approach

In the s u b s e q u e n t sections of this paragraph w e

will introduce the

nonlinear prediction problem in the following spaces: of the regular Volterra function~/ poIy/qomials; of the generalized coefficient-matrices;

~nd of

generalized z-polynomials.

2.3.1 S p a c e

o f tlae r e q u l a r V o l t e r r a f u n c t i o n a l . p o l y n o m i a l s

G i v e n the M - b l o c k , m - i n d e x e d matrix of the r a n d o m veu~iables [M} YNI-I

(expressed

by

(2.17) with

x=l

and

n=N-l) , ea~d a s s u m i n g

25 that the entries are linearly independent,

{ M}__I

A

Each

wil/

element from that s p a c e

M

where



I2nX~flto Lx-,n ~ - ~ n,n+l T

x~n

xtnl,n Ln+

LnX,fl,o

".__> Lx~n _ ~ .... n+l,n-i ~

Lx,n÷l n+l,n+2

Ln+ I,n+ l

Lx,n + n+l,n

Lnx'+~,n+1

Lx,n-I n,n+ I

Lx,n-I n+ l,n-I

Lx, n-i n+ l,n

LXP 0 n,2

LX, O n+l,o

LX, O n,n+l

f X~O

f

-

. .

-'~ Ln,n+l--~

n,2

Lx n,n+ i Lx

--~

n,o " t Lx+ l,n-i n~o

Lx

nsl

>

--~ L x sin

~

LX~O n+l,l

X~O

1'n+l,o +

Lx n,n+l

Lx+ l,n-i ntn

Lx n+ I,o ~

LX+iio

LX+ l,n n~n+l

Fig. 3.2 'Local' structure of the 'global' order-update index-set step.

n + n+l

47

o

0

t-" ILb;0"% k

•

..

ILb; 1

T[ •

f A

lb ° b ; 2 .............. % A

Lo

b| 3

f

•

A

,%

'

•

~

l

x=l

Ibm3

1

lLf; 2

-J

! x=2

x=3

3 ) Lf; 0

.J Initi~zations:

IL _

'I~Iew" elements:

T'

~

LX oeo

>

(x=O .....3)

5"i~.

3.3

n,n+l

(n=i,2,3 ~ x=O,.,.,3-n)

'Local' structure The symbols 0

cursions

a

of the third-order (N-3) indicate corresponding

of l~'ig. 3.2.

index-set recursions. 'local' index-set re-

48 From

Fig.

the L-forward

3.2 it f o l l o w s t h a t t h e will work

index-set

a)

initialization: L x n)o

b)

'uni-variate ) step: L x + n)o

C) )bi--vaLrlate' steps: L x

n)l

d) For

o)

)bi-variate'

d)

termination: L x)v n+l)v+l*

The

L- ~und B - b ~ c k w a r d

d)

for

follows:

-~ L x n)2 -~ "'"-~ nLX)n+l ~" L xn+l,o

we

steps:

i~U~za~ons:

x)v Ln)v+2-~

Lx)v n,v+2 x,v Ln,v+3

index-sets

L x+1'n-* ntm L x+l'n n,n+l

'uni-varla[e'

get:

(v=0) ....n)

x)v + step: Ln)v÷ 1

'uni-vari~te'

c)

recursion

Lx n) l

the B-for~vard index-sets

b)

b)

order-update

termination: L x n+l)o °

initializat[on: L x'v n,v+l

a)

~s

'global'

+ "'"

Lx,V x)v -~ Lx,v n)n+l ÷ n+1,o ÷ ...-~ L n + l , v + 1

are u p d a t e d

o.s follows:

( m = 0 .....n) (m=n+l)

stepS: L x+l'n-I ~. L x nim n)m+l L x+l'n -~ L x n)n+ l n+ l,o

'bi-variate' steps: L x + n,m+l

L x)O n)m+2

( m = 0 .....n) ( re=n+ i)

-~ L x'l n,m+3

+

"'"

-~ L x'n-I L x)n n + l ) m - i "~ n + l , m

termination: L x)n n+l)m"

We notice that e a c h is associated (N=3)

'local' order-update

with 'label-update'

index-set recursions The

index-set

n e a r ladder-filter

step. T h e

is p r e s e n t e d

recursions,

6dgorithms

step for the b a c k w a r d

of the third-order

in Fig. 3.3.

derived

presented

'Iota/' structure

index-sets

in this section, will underly

in the s u b s e q u e n t

paragraphs

nonliof

this chapter.

3.2 Nonlinear

filter a/~orithm: time-domain

In this p a r a g r a p h degree

we

will derive

nonlinear prediction problem,

of generalized

coefficient-matrices,

approach

a recursive

using projection

introduced

solution to the s e c o n d method

in p a r a g r a p h

in the s p a c e 2.3.2.

49 Let

{y}

denote a fourth-order ( M = 2 )

v e d o n the time-interval riables

[ 0,-l,...,-N ]

stochastic s e q u e n c e ,

and represented

yo,Y_l,...,y_N . C o n s i d e r i n g the index-sets

(3.2b), w e

define for

of the r a n d o m

n-0,...N

and

b y the r a n d o m

Lx n,m

(3.23)

va-

a n d L x'v ntm

the following submatrices

vorlatbles o.nd their products

Y-Jl

nmm yX

=~

(3.9)

=

n,m

2yx

n,m a n d the s u b m ~ t r i c e s L x'v . T h e n n,m

x=0,...,N-n

obser-

we

ly_j~

yX,V ntm

, expressed

(jl,J2) ,xnt m

by

(3.9) with

c o n consider the following ( 2

variance submatrices

Hx n,m

and

H x'v n,m

the former is given b y

le2HX n,m

H xn~m = E { ~ n , m

replaced b y

2)-block, multi-indexed co-

, where

lelHX

Lx nsm

] n,m

@ ~n,m } "

2 • 1HX

2 • 2HX n,mJ

ntm

Ik kk21 J12kl hJlJ2klk2

(Jl,J2,kl,k2)

a n d the latter is e x p r e s s e d

by

(3.10)

with

Lx n,m

replaced b y

(3.1o) ~ ~,m×L xn,m L x'v . n~m

Now let Ix

=

n~m

where

ljl

for

"~

[

(ki,k2)

~kl;j. I

[llX n,m

21x

] =

[i.

n,m

Jl

1..

(3.11~)

] ( j l , J 2 ) • L xn0m

JlJ2

• L xn~rtl

2o ]

~jlj2

[ lo

~

kzk2;jlj2 ]

(3.11b)

50 with

I0

and

20

ctively, w h o s e

being the o n e -

domains

Lx respectively. n,m |

Let us

and

~wo-indexed

are hhe uni- a n d

In a similar w a y

introduce

the

respe-

bi-variate parts of the index-set

we

following,

zero-malrices,

con

introduce

x-labeled

and

the mah-i~

Ix'v n~m

(x,v)-labeled,

sub-

spaces

I nx , m =

We

notice

{IX, mn

11'N-I -N-I,N

thai

by (2.37) with element

v

M.-2)

Fx nmm

}

will b e

se

n,m

domain

Ix'v ntm

is

Ix n~m

will b e

a two-block

F x'v nsm

(row),

with d o m a i n

x

expressed

multi-indexed

Gx

= Fx

n,m) ~x n,m

element

o n the s u b s p a c e s

.H x

n,m

n,m

x=0,...,N-n

the

to

(3.4~)

subsequent

and

(3.5),

subspaces

we

EMh

as

(3.12b)

coefficient-matrix w h o of the s u b s p a c e (2.39a),

(3.12a)

.%x

we

as

(3.13~)

n,m

X,V G X , V ~ --b~X'V°HX'V-G x'v Fn, m ' n,m j ]~x,v n,m n,m n,m ntm

According

(expressed

_o,N KN,N+ 1 °

will b e

D b ~ % v .. b x'v . Following n~m n0m

a family of inner-products

(Fn,m '

{ 2} K ~ _ I

fx ] n'm;jl-x'J2-x (Jl'J2) ¢ LXn,m

DF x - .Lx . Similarly, e a c h n~m nsm

will b e

introduce

v

precisely the s p a c e

of the s u b s p a c e

will b e

F x

.

n~m

while the 'biggest' s p a c e

b"Xn,m m [ f~n,m;Jl_ x

i.e.,

,xv

;

can

of the

(3.13b)

introduce

for

n=0,...,N

and

~o,N

-N,N+I

L-for-Nard

I~n~o = ~ (I~o) i

(3.14~)

51 B-forward

(for

v = O .... , n )

IX, v n,v+l

=

IX'v v { n,v+l }

(3.14b)

itx,n-i n,o

x,n-i = v { In, O }

(3.14c)

L-backward

(for

B-backward

m - i ....,n+l)

N x'n-I n,m

,, v { Ix'n-i } n,m

1[x'n n,n+l

3.2.1 'Local'

v

estimates

Denoting ta/dng projection subsequent

=

by on

,

xln {In,n+1}

and

,

m-.l .....n

(3.14d)

m=n+l

(3.14e)

errors

P l ; nx , m

("--I;n,m" p x,vj

the s u b s p a c e s

'local' estimates

and

the o r £ h o g o n a / ]ix n,m

errors

as

(Ix'vJ " n,m-

'

projection

we

operators,

will introduce

the

follows:

L-forward Following

(3.~a),(3.11),(3.12)

Itx n,o =

v

and

(3.14a),

we

{ix

x+l,n-i ' In-i,n }

can

rewrite

a~

Itx

n~o

(3.i5a)

A

We

define

the L - f o r w a r d

n-th o r d e r

p

estimate

x÷i,n-i i 11;n-l,n

x

'ix

n,o

of the

ix+l,n-1 n-l,n

i

x

as

(3.15b)

52 Let

0jl

denote the zero-entry with 'coordinate'

the L-forward ^ n-lh order approximation error, c o r r e s p o n d i n g to the estimate ix , will n,o

be expressed

as

x =A p l x+i,n-I i - I - [0 An,o --~ ;n-l,n x x o

(since the estimate x ~n,o). This

pace

Jl " T h e n

'%1 x nto

~x I ± n,o ~

is c o n s i d e r e d h e r e ~

~x+l,n-I n-l,n

(3.16a)

etn element of the s u b s -

error caxl b e rewritten in a renormalized form

AX n,o "

x An,o

x

C:

1{ An,o [

=

n,O =

[x

ax ] n,OlJl-X,J2-x (jl,J2) • L x n~o

n,o;Jl-X

in accordance

the errors

with

(3.16)

respec~vely, if

B-forward Using

(3.12b).

can

observe

are precisely the

M~2

( for

We

, x=0

and

that the

estimate

{M) IN;o , { M }A N

~d

c a n rewrite the s u b s p a c e s

I

v { Ix, x

'

Ix } n,o

n,v+l

Let

'

X x'v n,v+ 1

if

Ix'v-I } n,v

car* introduce the B - f o r w a r d

0jl,j2

A

i

(2.54)

(3.14b)

as

v=0 (3.17a)

{v { iX,x+v [

ix, v

{M)A N

and

v=0,...,n)

(3.4b), w e

we

(3,15b)

n=N.

Xx'v s < n,v4-I I

Then

(3.16b)

p I;n,o x Ix,x

c

if

v--l,...,n

estimates

Itxn,o

if

v=0

IIX'V-i n,v

if

v--l,...,n

(3.17b)

p

x,v-i 11;n,v

i

X,X+V

E

stand for the zero-entry- with 'coordinates'

(jl,j2) . T h e n

53 the B-forward

approximation

will be e x p r e s s e d

j

A =

n,'%r%'- ~

to the estimates

(3.17b),

a~

. p±

ax,v

errors, c o r r e s p o n d i n g

x

a

Xx

I[ ;n,o Ix,x

if

v=O

if

v=l,...,n

n,o

(a.18a)

/

[ p l x,v-i

i

1

ITX'V-I n)v

x)x+v

H ;n)v

i.e., Ax ' v

nsv+ 1

as the estimates B-forward

-

Ix,x+v

~x,v n,v+l

(3.18)

errors

AX'V = n,v+ 1

=

[0

~X)V ] n,v+ 1

O)V

are treated here

can

be e x p r e s s e d

(3.18b)

as elements

of

x,v I[n,v+i). The

in the renorma/ized

form as

ax'v llAX'Vl[l-lnv+ n,v+ 1 , llx,v n,v+ 1 x,v

(3.1So)

x,v

= [ an,v+l;Jl-x

an,v+l;Jl-x,J2-xl

(jl,J2)

x,v £ Ln,v+ 1

(3.14c), using

(3.5a), as

L-backward Let us

K~;n-i nuo

rewrite

n" x , n - 1

~' V {In~.q:nl -- )

lq)O

so

that

we

will define

-1 ~x,n n,o

The

n,o

A

x,n-1

= P~;n-l,n

L-backward

k p x,n-i = R;n-l,n i x + n

L-backward

B x,n-1

the

approximation

lx+n

=

-

ix+n}

estimate

(3,19a)

as

l[x,n-i n-l,n

e

error

:1. x+n

!

will then

Vx'n-1

[ln,o

(3.19b)

be

On]

i

I x'n-1

n-l,n

(3,20a)

54 or, in a renormalized

B x'n-I n,o

form,

B x'n-1 IIB x'n-1 -i n,o n,o ]l~x,n-i n,o

=

-- [b x'n-1 n'°;x+n-Jl

(for

B-backward

F'ollowing (3.5)

Hx'n-I

m = l ....,n+l)

and

V

(3.14), w e

{ Ix'n-I

c a n write

'

i x + n + l-m,x+n}

'

mml"°°tn

(3.21a)

x~n . ..xln--I Kntn+l , v [in~ n

,

ix,x+n }

,

m=n+l

(3 21b)

"

the B - b a c k w a r d

~x,n-1 n,m

~= _ x,n-I P~;n,m-i

ix, n A= P x,n-i n, n+ 1 ]I;n,n

Consequently,

estimates will be e x p r e s s e d

C

B x,n nmn+l

IIx'n-I n,m-i

,

ix, x+ n

Nx'n-1 n, n

'

the B - b a c k w a r d

c

approxima£ion

;n,m-I i x + n + l-m,x+n

A= Pli x,n-1 ;n,n

as

i x + n + l-m,x+n

i

x,x+n

±

!

~[x, n-i n,m-1

ix, n-i n,n

i

m=l,...n

(3.21c)

m=n+l

(3.21d)

errors will be defined as

,n-1 n,m

so

(3.20b)

n,m-i

n,m

Hence,

bY.n-1 ] n'°;x+n-Jl'x+n-J2 (jl,J2) E L x'n-I nlo

'

m=ii...,n

(3.22a)

m=n+l

(3.22b)

that Bx,n-1 n,m

~,n

~n,n+l

-

[~x,n-1

= i x + n + l-m,x+n

=" lx,x+n

-

r~ ~ ' n

~ n,n+l

n,m

0

o,n

On+ i-m,n ]

(3.22c)

]

(3.22d)

55 "Dhe B - b a c k w a r d

errors c a n b e e x p r e s s e d

in a renormalized

illx, n_l-i

Bx'n-ln, m I1 Bx'n-ln,m

Sx'n-ln,m "

form

,

m n l .....n

(3.22e)

,

m-n+ l

(3.22f)

n~m Bx,n

x,n n n,n+1 .. Bn, n+ I U 5~n,n+111 -* ~x~n n~n+l

similarly as the L - b a c k w a r d

3.2.2 D e c o m p o s i t i o n

PoUowing

error

of s u b s p a c e s

B x'n-I nmo

(3.20b).

'

the considerations

consider the 'local' decomposlf/ons

o f the previous

paragraph,

we

can

of s u b s p a c e s :

L-for~rard Since

Ax n,o

is in

from (3.16), w e

Ix n,o

'

but is or%hogonal to

n,o

=

fix+ 1,n-I

n-l,n

•

x

v { An, o}

w h i c h implies the 'local' decomposition

x . p x+1,n-1 PII;n,o K;n-l,n

x

P~n,o

denotes of

B-forward

v=O,...,n)

Since

+

(3.23a)

of projectlon operators

x PA;n,o

(3.23b)

the om£hogone/ projection operator, taking projec-

£ion o n the s p a n

(for

, as it follows

c a n write

Kx

where

~x+l,n-i n-l,n

Ax n~o

AX'Vn,v+l belongs

to

~n,v+X'V1

but is orthogonal to:

]iXn, o

(if

v=O),

56 and to

I x'v-I n~v

Ix'v

(if

v--1 ..... n ) ,

I IIx n,o

=

in accordax,c e witkl ( 3 . 1 8 ) , we o b t a i n

o

Ax, o v{ n,l }

v~O

~

x,v v { An,v+ I}

v=l,...In

(3.24a) n,v+ I

This

11x'v- 1 n,v

implies

p

xtv Pl;n,v+ 1

p

where

< l

xtv A;n,v+l

X l[;n,o

+

,

V~0

,

v=l,---,n

(3.24b) X,V--I ][;n,v

p

+

is the projection

operator

on

the s u b s p a c e

spanned

by

Ax, v n,v+ i " L-b ac k w a r d Observing

h a l to

that

I xn-l,n 'n-1

B x'n-I , see

(3.20),

I x'n-I n,o

resulting

PB;n,o

IB-backwvi~rd From

we

to the s u b s p e t c e

]Ix'n-I

n~O

but is o r t h o ~ o -

car* write

= ITx'n-1 n-l,n

(9

v

{B x'n-I n,o }

(3.25a)

in lhe d e c o m p o s i t i o n

p

v~here

belongs

n~o

x,n-i ~;n,o

_ x,n-i = ~]l;n-l,n

is the projection

( for

(3.25b)

+

opereltor o n

the s p a n

of

Bx'n-ln,o "

m=l,...,n+l)

(3.22)

it follows

that

]ix,n-1 n,m

= ~x,n-1 n,m-i

•

v {Bx~n-1 } n~rfl

,

m=l,...,n

(3.26a)

57 ix, n = I x'n-I n,n+ I

This

•

V {B x'n . }

m=n+l

(3.26b)

imp)/es

p

x,n-1 ]I|n,m

p

x,n

= o x,n-1 --~;n,m-i

+

x~n-i I[;n,n

+

~

p

W;n,n+l

p B ; nx,n-i ,m

where spanned

by

We

can

x,n PB;n,n+l

n +i ) ( P B ; n~tn

B n x'n-I .m

3.2.30rthonormal

(Bx:nn+ I )

that the L - a n d

In o r d e r

to s h o w

A x'u n,u+l

'

that, let u s

a/qd let u s

m=n+l

~sume

belongs

£o

we

can

that •x,v n,v+l

show

A x'v nlv÷1

B-forward

set in the s p a c e

consider

A x'v n,v+ 1

Consequen~y,

,

operator

(3.Z6c)

(3.26d)

on

the s u b s p a c e

ba~es

observe

In a similar w a y

m = l,...,n

"

(v--0,...,n) f o r m the O N

A xn ', vv + l

,

is the projection

A x'v n,v+ I

and

,

two

~

errors

A x'u ± n,u+l

a2ud the matrices.

A Xn,,Vv + l Ix'u-I n,u

and

D N x'v n,v+l'

v=0,..,n

A x'u n,u+ I

that for

±

obtain for

A x n~o

of g e n e r a l i z e d

B-foFwca'd

v < u. S i n c e ' we

errors

( 3.27 a)

v:O,...,n

A x n~o

(3.27b)

the entries of

A x = n will f o r m the O N

[A x

n~o set.

AX, o n~o

"'"

An,n x • n - 1 A nx:n ,n+

11

(3.28)

58 If w e

introduce, a c c o r d i n g to (3.15a)

[Ix

iX, x

...

and

(3.1?a), the following set

Ix,x+n_ 1

ix, x+n]

(3.29)

J/~en (3.28) wiU be the orf/,1onormalized version of that set. Using (3.28) we cain write the following 'global' orthogona/ decomposition of subspaces

x,n

~n,n+ i

:

f An,n+l

Bx Ax

xiO Bn+l,l

. AX, v n,m

AX,V n,m+l B x,v-I n,m

Fig. 3.~ 'Local' structure of the 'global' section of the timevariant nonlinear ladder-filter.

88

"l

,--Bo°(Z)-~

¢---

B

(

'(z)-m

[I

~~(z)

--J

u ---.~ ~a(z)

I

x--1

J ~(z) x=2

J ~3(z) o

x=3

> Initia/izations:

lu

~=

T

.)

'New' elements:

B2;2(z)

'

ao~,o(Z)

u

Bx'n _ (z)

'

n ~ n-l- 3

>

x=O,...,3)

Fig. 3.5 'Local' structure of the third-order (N=3) time-variant prediction filter. T h e symbols ponding 'local' 8-recursions of Fig. 3,4.

(n---i,2,3 ~ x=0,.,.,3-n)

nonlinear (Levinson) O indicate corres-

89 Let u s nested

observe

'local' 8-transformations

the s u b s e q u e n t m6dn

that the filter structure consists

sets of the forward

mutually orthonorma/

hal' order-update solutions

(actually - ~ i v e n ' s

step

in Figs,

a.s we]/ ~

after e a c h

(see

3,4: a n d

3.5). T h i s

means

of the 'local" ol~hogonal

Vieira a n d

De~vilde

(1982)).

cients

is associated

Kailath

These

(with n o r m s

(1978),

matrices

being

will re-

after e a c h

'~lo-

resp. horizontal

orthogon~9/it'y requirements, a/nd norma/ized

aJnd D y m

are specified b y

less than one),

errors

with the J-lossless

Dewilde

s o that

that the structure of this

as the filter consists

Dewilde,

backward

the subsequent-vertical,

will satisfy the desired

section

rotations)

'local' a/qd, hence,

nonlinear ladder-filter

sections. E a c h

of a cluster of

'local'

8-matrix

(see

(1981,1984),

the reflection coeffi-

actually the Fourier

coeffici-

enhS. Since the p a r a m e t e r observation, ar prediction

the reflection coefficients x , being the b a c k w a r d which

We

can

input s e q u e n c e

notice t h a t ON

ar prediction p r o b l e m

that

interpreted

(not Toeplltz)

well a~ back~vard

obtained

be

to the generalized

in meprettere

and

Lie

be solved

of the generalized the s p a c e

is reflected in the H e r m i cova/'iar*ce matrix. computes

the forward

~s

the solution to the N-th order nonlinein the innovations

context)

is

@/so r e m a r k

terms, the filter of B~i~. 3.5 Will immediately

(time--vari~/nt) linear L e v i n s o n

filter c o n s i d e r e d

(1980).

~Ve car, c o n c l u d e lem c a n

, and

'level' in the filter structure. W e

ne.qlectin~ all nonlinear

reduce

{y}

(actually c o n s i d e r e d

at the 0-1abeled

point of

is implied b y the nonstationarity

the nonlinear ladder-filter and

on

~s the 'current' time, this nonline-

of the genera/ized

b~es,

)galns )) d e p e n d

shift from the reference

filter is time-variant. T h i s

of the higher-order tian propet~y

(i.e.)the f i l t e r

Uqat the nonlinear

geometrically, (block,

using

multi-indexed)

of the genera/ized

least-squares

projection

method,

prediction

in the s p a c e

coefficient-matrlces,

(block, multi-variate)

prob-

and#or

z-polynomials

in

(provi-

9O d e d the higher-order c o v a r i a n c e

dmt~ or, equdva/ent|y, the higher-order

spectral functions of the underlyln~ s £ o c h ~ t i c former a p p r o a c h fine M - D

results in the o p t i m u m O N

impulse r e s p o n s e s

sequence

approximation of the set of

of the nonlinear prediction filter of the Volte-

rra-Wiener class. In the latter case, the o p t i m u m O N mation of the set of M - D

polynomial approxi-

transfer functions is obta/ned. W e

the results p r e s e n t e d he~'e ~ e s e n t e d in Z a r z y c k i

are given). "imhe

and Dewilde

equlv~/ent to [he algebraic solution pre(1983a),

a n d to the ~eometric solution

of the stochasf/c esf/mation problem, d i s c u s s e d in Z a r z y c k i

The

non]/near a p p r o a c h

(1984a, b)o

to the lea.st-squares prediction p r o b l e m

(for higher-order stoch~.~tic s e q u e n c e s ) the linear treatment)

r e m a r k that

may

estimation a c c u r a c y

result in better (than in

(if the s e q u e n c e

is n o n - G a u s -

sign), ho'vvever, complexity of the genera]/zed nonlinear filter p r e s e n t e d here [ n c r e ~ e s

rapidly ( s y n c h r o n o u s l y

step), a n d b e c o m e s

rather big e v e n

with e a c h

'global' order-update

in re~tively low-order nonlinear

filters ( c o m p a r i n g to the complexity of the linear filter), as it c a n b e seen

in Fig. 3.5. "l~herefore, the complexity reduction p r o b l e m will b e

the subject of the next chapter, w h e r e

time-invariant as well as 'quasi-

linear' ladder-filter algorithms will b e presented.

4. T I R I E - I R V A R I A N T

We

AND

noticed in the

'QUASI-LINEAR'

~DER-F'ILTERS

previous chapter that complexity of the ~enerali-

z e d nonline~gx' ladder-filter i n c r e a s e s

rapidly ( s y n c h r o n o u s l y

'global' order-update step), a n d b e c o m e s nonlinear filters. Consequently,

with e a c h

rel;~tively 'big' e v e n

in this chapter w e

in low-order

w i s h to c o n s i d e r the

problem of complexity reduction in nonlinear ladder-filters. In order t o obtadn efficient nonlinear filter algorithms, w e

w i n first d i s c u s s the nonlinear

least-squares prediction problem for stationary (in the higher-order s e n s e ) stochastic s e q u e n c e s . W e

~Nill s h o w

near Ume-invariant filter w h o s e

that the solution results in the nonli-

complexity is m u c h

reduced

(comparing

to

the genera/ized algorithm). Purther complexity reduction will b e a c h i e v e d by introducing simplified nonlinear estimation s c h e m e s ,

ca/led 'quasi-linear'

filters a n d associated with the o p t i m u m prediction of higher-order stochastic s e q u e n c e s

whose

'distar,ce' from the G a u s s i a n

s e n s e to b e defined). T h a t Zarzyckl and DewJ/de

problem

(1983b),

has been

is l o w

(in a

introduced algebraically in

and considered

ce of the Volterra functional polynomials)

sequence

geometrically

in Z a r z y c k i

(in the s p a -

(1984c,e).

4.1 Shift-invaria~nce of inner-products

Let u s

a~sume

stationary (in a w e a k

Ulat the underlying stochastic s e q u e n c e four~h-order s e n s e ) ,

Then,

{ y }

is

following (2.19), w e

92 will

obtain

I-Ix n,m

H x'v n,m

regardless

of t h e x - s h i R

with d o m a i n s

respectively.

- H e = "9 n,m n,m

-- H x + l ' v , , n,m.

are the g e n e r e d i z e d ces

= H x+l n,m

nom

Applying

T v n,m

(4.1b)

(i.e., the time-shift), where

(block, DT

H O'v ~ n,m

(4.la)

multi-indexed)

= L° x LO n~m n~m

(4.1a)

x x (b-~n,m ' G n , m ) xx

Toeplltz and

in (3.13),

we

"1"

n,m

covariance

DT v n,m can

and

= L °'v nlm

T v n,m

subma%rix L O'v nlm

,

write

= (_x+l _x+i~ - b ' n , m ' ('Zn,m) E x + l ~"

n,m

n,m

=

(FOn, m ,

A

F

G°m)

.T n,m

=

o

.~_T nmm

n,m

= ( F ~ , m ' Sn,~)~o

(~.2~)

n~m where

F

[fn,m;jl

=

f . . ] L° n'm;Jl'J2 (Jl'J2) ~ n,m

n,m

Applying

(4.1b),

we obtain similar relations

for t h e

(x,v)-labeled

(4.2b)

quanti-

ties

.,v

x,v

~ Fv

(Fn,m'Gn,m)ix,v

.~ v

n,m

.Sv

n,m

n,m

v

n.m "

(Fn, m ' G n ,

v

(4.z~) m)i[o,v n,m

wiLh

F v = l-l,m

[ fv . • n'm;Jl

fV . . ] L O'v n'm;JlJ2 (Jl']2) ¢ n , m

(~.2d)

93 Equations product next

(4~.2) e x p r e s s

the x-shift (i.e., time-shift)

in the h i g h e r - o r d e r

paragraph

simplifications

we

(i.e., fourth-order)

will show

of the nonlinear

4.2 "l~ime-lnvarlan% n o n l i n e a r

Following

Ao

n,m

A A

~__-

a

significant

inner-product

(4.2), w e

will satisfy the following

=

.

]

.

n'm;J1]2

notice

relations

[a v n'm;Jl

(¢.3a) (jl,J2) c L °

n,m

(v=O,...,n), w e

: A °,v n~m

Similarly, for the L- a n d

obtain

=

a

] n'm;]lJ2

B-backward

(jl,j2) E

errors,

we

L O'v n,m

can

write

= B x + I , n-I = B°, n-I = n~m n~m

A Bn-1 n,m

m=0~...,n , a n d

~':,~ n,n+l

in

=

errors

A x,v = A X + l , v n,m n,m

if

Of the

errors

n'mlJl

the B - f o ~ a r d

Bx, n-I njm

will result

In the

ladder-filter e d ~ o r i t h m

[a

==

A A v : n,m

property

case.

n,m

n,m

For

stationary

of inner-

ladder-filter algorilhm.

approximation

~ Ax+I=

n,m

this

the shift-invariance

that the L-forwc~rd

Ax

that

invariance

[bn-I n,m;n-Jl

for

~ Bn n,n+l

bn-1 n,m;n-Jl,n-J2

(4.40.)

] (jl,J2) ( Lo,n-i n,m

m-n+l

-_ [ b n , ~ + l ; n _ q ~

bn . . 1 n,n+ l;n-] l,n-] 2

(jl,ja)

e Lo, n n,n+l

94 Consequently,

the forward a n d b a c k w a r d

'global' O N

bases

will

satisfy

A x = Ax+£= n n

A ° £ A n n

(4.5a)

B *n - ~ +n * =

B ° ~: B

(4.5b)

n

n

wihh

A

B

We

n

-

n "

[A

A ° n,l

n,o

[ 13 n - I

n,o

"'"

"""

A n ntn+l ]

B n-I n,n

B n n n +.I]

notice that in stationary case, the entries of

initializations in the 'g/obal' order-update step higher-order forward a n d b ~ c k w a r d

solutions

(4.5c)

(4.5d)

B n

n ~ Z~n+ l

will be u s e d

n+l

as

, yielding the

and

l~n+ 1 . C o n -

sequentiy, in the stationoxy case: a)

there

is n o 'nesting' b e t w e e n

the x-labeled 'levels' in the structure of

the nonlinear ladder-filter; b) the nonlinear filter atgorithm c a n b e e x e c u t e d

at e a c h

x-labeled 'level'

s ep arately; c)

it is sufficient to run the a/gorithm at the (x::0)-labeled 'level' only,

following (4.5). Hence,

the stationary version of the generalized nonlinear ladder-filter

algorithm -#vil/b e obta/ned if w e

c o n s i d e r the 'loca/' LL, LB, B L

recurslons ~t the (x=0)-labeled 'level'. F o r sion of the L L "local' ordeP--updatte recursion

An,1 = ( 1 - [ P n , 1 1 2 ) - ½

Bn, I

( l - [ p n,1] 2) -P~

([An, o 0n+l]

(-Pn,l[An,o

and BB

example, the st~ttionary ver(3.42) will take the form

- Pn,1 [ 0o Bn-1])n,o

0n+l ] +[0o

B nn,o -l])

(4.6a)

(4.6b)

95 with

P~,I" ([A,,,o o . i] . [0o Bn-:t])Io~,o

(4.6~)

n,1 The ssed

transform-domain as

counterpart of the L L recurSion

(4.6) will b e expre-

(following (3.68))

InlzI 0. Fnozl

(4,7~)

with

69 n,l =" (I- [Pn, l]2) -½

(4.7b) -P n,l

and ~,i

The

remaining LB, B L

tionary c a s e

"

(A~,o(Z)'z'~.oi(z))z

and BB

'local' order-update

will be the 'local' recursions

(4.7=) recursions

of A p p e n d i x

in the sta-

2, provided the

x-label is r e m o v e d . The

L L 'local' recursion

[ion of the corresponding

(4.7)

cart b e interpreted a~ the L L - s e c -

nonlinear ladder-filter

B~,I(Z) A o(Z)

~

-- A i(Z)

B~-i(z) n

(~.Td)

g6 "l~he remaining

'local' LL, L B

and

filter will again b e

expressed

v i d e d the x-labels

~/'e r e m o v e d .

BB

sections

of the stationary nonlinear

a s the 'local' sections These

of A p p e n d i x

sections, c o n n e c t e d

together, w i U constitute the 'global' section

accord[n~y

n÷l

of the filter. W e

notice that the "set o f reflection coefficient~ c o m p u t e d

b y the alsorithm

the 'global' step x-labels Pn,m

n -~ n + l

are again

removed.

P vn,m

~qd

paraJneter

x

, will b e

(being

Observing

me

(N=3)

rithm c ~ n

can

observe

~s

lying stochastic On

sequence

(block,

mu/ti-indexed)

moving

all nonlinea~r terms, w e

sequences,

~

Toeplitz

considered

Comparing (Fig. 4,1)

in the time-variant

of the generalized

Kailath

(in a w e a k

higher-order

Levinson

matrix. W e

(1982),

Vieira a n d

Deprettere

ladder-filters, has

been

be con-

also notice that re-

a~nd Lie

we

achieved

ase

with e a c h achieved

sense).

the classical linear stationary

i~[ai%ath (1978),

of the time-variant

of the quaint[ties p r o c e s s e d

WiLl b e

the u n d e r -

algorithm c a n

will immediately o b t a i n

in Dewllde,

algo-

factorization of the generalized

Uqough, the n u m b e r

ty reduction

ca~e.

a/qd/or of the

prediction filter for s e c o n d - o r d e r

nonlinear

synchronously

matrices

provided

covari~nce

the structures

tion of the filter complexity

is

or%hogonaliza-

z-polynomials,

for C h o l e s k y

(Levinson)

(1981),

'local'

nonlinear ladder-filter

for ( G r a m - S c h m i d t )

the stationa/q~" nonlinear

a~ the fast m e t h o d

variant

considered

is stationa/'y

sidered

time-invariant A R

o n the

conclude

is time-invariant. T h e

t[me-invariant

of the generalized

the other hamd,

a.nd D y m

o n 'current' time), w e

b e treated as the fast m e t h o d

in the s p ~ c e

do not d e p e n d

that the time-invariant nonlinear ladder-filter

tlon of the ba.sis in the s p a c e basis

the

notice that the filter satisfies precisely the sa-

or£hogonality requirements, We

(3.43), p r o v i d e d

at

that the reflection coefficients

prediction ladder-filter

in Fig, zi.l. W e

by

actual/y the filter gains)

structure of the third-order presented

expressed

(i.e., they d o not d e p e n d

that the nonlinear

n +

2, pro-

Dewilde

(1980).

(Fig. 3.5)

and

time-in-

notice that significant reducin the stationary case,

al-

in the filter ~ill still incre-

'global' order-update

step. Further

in 'quasi-linear' prediction filters.

complexi-

have

structure

All s y m b o l s

F i g . 4.1 ' L o c a l '

III

IL]I the s a m e

meaning

of t h e t h i r d - o r d e r

time-invarlant nonlinear a s in Fig. 3.5.

(N=3)

ladder-filter.

(D

98 4.3 'quasi-linear' ladder-fiite[s

In this p ~ r ~ g r ~ p h ladder-filter

algorithms

we

w J ~ consider

which

we

ters will yields better estimation le their complexity considered

nonllneam

Let u s ce

{y }

riables

will b e

a~sume

accuracy

(than in the linemr cruse)

in c o m p a r i s o n

filwhi-

with the previously

algorithms.

that

is r e p r e s e n t e d

of simplified nonlinear

w J ~ c~i[ 'qu~si-lhqear' filters. T h e s e

reduced

filter

a class

the underlying

fourth-order

b y the following s u b m a t r i c e s

stochastic

sequen-

of the r a n d o m

va-

( a n d their productS)

yl,n n,n+l

.

__

_

n=0,...,N-i

Y-JlY-J2J

(4.8a)

(Jl,J2) ¢ L n,n+ l,n 1

where L l'n = LI 2Ll,n n,n+ 3. n u n,n+l

with

LI = n

{ 3,...,n+l }

2Ll'n n,n+l

Now win~

let u s

(4.Sb)

and

= sym2L I n

introduce

for

× sym2Ln1

n=0,...,N-1

(4.8c)

and

~ =0,...,n+l

the foUo-

index-sets

~(~) n

& L I u 2L(~) n

n

(4.9~)

99

where

the bi-varia~e part of the i n d e x - s e t

2L(n~)

"l~hen w e

if ~ = n + l

- i f

If w e

0




Lx

n=m

Lx

~

nln+l

f

LX+l,n n,n+l

Lx

n+ljo

(A.23~)

128 BL-recursions:

'loco/' order-upda£es

for the 'uni-variate' part of the

B-fo~va/~d index-se£s, a n d for the 'bi-varia£e' part of %he L - b a c k w o 2 d index-se~

i.e., for

v=0

n~l L x'O nt2

=

L x'O n~l

u

Lx n,l

.

{x,x} %,....

u Lx U {x+n+l} n,o ~, 2

( A . 2Zia)

LX~ O n,l

a n d for

v=l,...,n

Lx,v-I n,v+ 1 LXt v

n,V+2

= L x'v L x'v-I ~ { x , x + v } n,v+l U n~V+1

u Lx, v-I ntv

u {x+n+1}

(A,2a~b)

T LX, v *%~V+ i

These

recurs[ons will b e in£erpre£ed as the B L

Lx, o n, 2

Lx~° n,l

~I .~

LX, v n,v+ 2

~ r

LX'° n,2

Lx'v n,v+l

'"

Lx n, 1

BB-recursions:

index-set sec%ions

~ "~

"2

(A.2~c)

LX, v 2 n,V+

Lx, v-I rl,v+1

'local'

order-updates

B-foF%va~d a n d B - b a c k w a r d

for

the

Jbi-v~iate

index-sets; i.e., for

v=0

~ parts

and

of

the

m=3,...,n+3

129 LX

n,m-1 Lx. ° _- Lx, ° n,m n,m-I

u

A

f

Lx n,m-I"

{x,x} k

...............

u L=,m_ 2 U {x+n+4-m,x+n+l} J

%

(A.25a.)

Lxs O

n,m-i

arld for

v~.l,...,n e ~ n d m m v + 3,...,v+n+ 3

ijxlv'l n,m--i [ L x'v n,m

m

L x'v n,m-I

u L xn ,' vm--12

- { x,x+v}

U

•

u {x + n + 4 + v - m , x + n + 1 }

Y LXl v

n,m-1

These

recursions will result in the B B

index-set sections

L~,o

LXi v r'isi-n

ntm

Lx, O n,m-1

~ ~

(A.e5b)

>

L x'° n,m

Lx ' v n Im - 1

-~ "~ |

Lx n,m-i

Lx,v-I n,m-I

~

Lx'v n~m

(A.25c)

130

'LOCAL' 0 R D E R - U P D A T E

LB-recursions:

for

RECUI~SIONS

m=2,...,n+l

A~,m_z(z) ]

AX, re(Z)

lB~,m(Z)J =

X

n,m

and for

,.

@

x n,,m

(A~,m(Z)

(A.26a) z.Bx+I,n-L(Z) [ n, m-i ~ j

x+i,n-l.

, Z'Bn,m_ l

(Z))Z

(A.26b)

re=n+2

(A. 26C:)

Sn,n+i(z) (A.26d)

x P

BL-recursions-

for

n+l,o

v=O

~"~(~)] . 0~.o XsO = Pn,2

( A X~, , I (O z)

-A::~(=~I

, B~,I(Zl) z

(A.2Va)

(A.27b)

131 for

v=l,...,n

[]:::+~(~1 .o.,v [ n,v+2

px,o ~ n,m

BB-recursions:

for

v=0

(AX, O (Z) n,v+l

and

(A.27c) B x'v-I ( Z ] I n,v+ 1 ~

"J

B x'v-I (Z~) n,v+l ~ " Z

'

(A.27d)

m=3,...,n+3

Ax'O ~ (Z)l n,m-I | =