Journal of Shanghai University (English Edition), 2004, 8(4): 391--396 Article ID: 1007-6417(2004)04-0391-06
A Backward Stable Hyperbolic QR Factorization Method for Solving Indefinite Least Squares Problem X U Hong-guo ( ~, ~,, ~ ) Department of Mathematics, University of Kansas, Lawrence, KS 66045, USA We present a numerical method for solving the indefinite least squares problem. We first normalize the coefficientmatrix. Then we compute the hyperbolicQR factorizationof the normalizedmatrix. Finally we compute the solution by solving several triangular systems. We give the first order error analysis to show that the method is backward stable. The method is more efficient than the backward stable method proposed by Chandrasekaran, Gu and Sayed. Abstract
Key words indefinite least squares, hyperbolic rotation, ~ p.q-orthogonai matrix, hyperbolicQR factorization, bidiagonalfactorization, backwardstability. MSC 2000 65F05,65F20,65G50
1 Introduction We consider the indefinite least squares (ILS) problem m i n ( A x - b ) T~,,p,q(Ax - b ) ,
(1)
X
where A E R Cp+q)×~ , b E R P÷q, and ~,,p,q =
0
-Iq
is a signature matrix. This problem has
several applications. Examples include the total least squares problems ( [ 1 ] ) and the H" smoothing problems ([2,a]). It is known that the ILS problem has a unique solution if and only if AT ~,p,qA > 0 ,
(2)
e . g . , [ 1 , 2 , 4 ] . In this note we assume that the condition (2) always holds. Note that under this condition we have p ~ n . The ILS problem is equivalent to its normal equation
A T ~ p , C 4 x = A T~,p.qb.
(3)
Since the normal equation is usually more ill-conditioned than the ILS problem, numerically, one prefers Received Dec. 1,2003 Proiect partially supported by the National Natural Science Foundation of China (Grant No. 0314427) and the University of Kansas General Research Fund Allocation (Grant No. 2301717) XU Hong-guo, Asso. Prof.,
[email protected],edu
to solve the problem by working on the original matrix A and the vector b directly. A typical example is the method that uses the QR factorization to solve the standard least squares problem (which is the special case of the ILS with q = 0 ) , see, e . g . , [ 5 , 6 ] , and [7,Sec. 5.3]. Following the idea of the OR factorization method, recently two methods were developed to solve the general ILS problem. The method proposed in [ 1 ] uses the OR factorization of A to solve the ILS problem. The precise procedure is as follows. First compute the compacted QR factorization
A =QR, where Q is orthonormal and R is square upper-triangular. Then compute the Cholesky factorization
LL w = QT~,p,qQ. Finally compute the solution x by solving three triangular systems successively,
Lz = QT~,,p.qb,
L Ty = z ,
R x = y.
It is easily verified that the solution ~¢ satisfies QT~_~p,qQRx = QW~,,p.qb. By pre-multiplying R T, it is just the normal equation ( 3 ) . In [ 1 ] it is shown that this method is numerically backward stable. The method proposed in [4] uses the hyperbolic QR factorization, an analog of the QR factorization, to solve the ILS problem. Definition 1
Let ~ p,q = [ "0
- 0Iq l
Journal of Shanghai University.
392 (1) The matrix H E R ( p + q) × ( p + q)
is
~ p, q-orthogo-
nal or hyperbolic if H T ~ p,q H = ~ p,q. (2) Let A E R (P+q)×'~. The factorization
where U, V are orthogonal and D is upper-bidiagonal. Compute d l = UTbl. Step 2 Solve the matrix equation
SD = A2 V
is called the hyperbolic QR factorization of A if H is N p,q-orthogonal and R is uppertriangular.
for S. Step 3
The method given in [4] consists of the following steps. First compute the hyperbolic factorization
f = [0
Compute In
- s T ] [ dl b2 1"
Compute the hyperbolic QR factorization and simultaneously update the vector
g= [In 0]/-ITNp,qb. Then compute x by solving the triangular system R~=g. This method is very similar to the QR factori~tion method for the standard least squares problem. Unlike the first method, which still needs to work on the product QT Np,qQ, this method directly work on A and b. Therefore it is less expensive (e.g. , [ 4 ] ) . In [4] it is also proved that under some mild assumptions the hyperbolic QR factori~tion method is forward stable. However, it is not clear whether the method is also backward stable. The main problem is that one can only show that the computed hyperbolic 0 R factorization and vector g satisfy a mixed backward-forward stable error model ( [ 4 ] ) . In this note we combine the ideas that were used for the previous two methods to develop the third method. We will also use the hyperbolic QR factorization. But for numerical stability we will compute the hyperbolic QR factorization of a normalized matrix. The general procedure of the method is given in the following algorithm. Algorithm 1
b=
[bl]ER"q b2
Given A = [ A A21 ] E R ( ' + q ) × n
,whereAIER'×"
,A2ERq~"
an d
,
b i E R P, b z E R q, and A T ~ p , q A > O , the algorithm computes the solution of the indefinite least squares problem (1). Step 1 Compute the permuted hidiagonal factorization
where H is ~ n,q-orthogonal and R is upper triangular. Step 4 Compute y by solving the triangular systems successively,
RTw = f ,
Rz = w ,
Dy = z.
Compute z = Vy. We will discuss the detailed computation process in the next section. In section 3 we will give the first order error analysis and show that the algorithm is numerically backward stable. For error analysis we will use the standard model of floating point arithmetic ( [8 ,pp. 44] ) :
f l ( a o b ) = (a + b ) ( l + ~), f/(~/-a ) = ~/-a (i + $),
I~l~ 0. Obviously the positive definiteness of R r R S r S is preserved during the reduction process. From this one can verify that I xkl < 1 for all l ~ k ~ n . So the algorithm will not break down. We discuss the cost of Algorithm 1. Step 1 It needs about 4 p n z - 4 n 3 / 3 flops for the bidiagonal factorisation. If the method in [9 ] is employed, it needs about 2 p n 3 + 2 n a flops. Computing dl needs about 4 p n flops. Step 2 It needs about 2qn 2 flops for computing A2 V, and about 3qn flops for solving the bidiagonal system for S. Step 3 It needs about 2qn flops for computing the vector f . It needs about 2qn 2 flops for computing R . The hyperbolic matrix H does not need to be updated. Step 4 It needs about 2 n 2 flops for solving three systems of equations to get y. Finally computing x needs about 2 n 2 flops. Table 1 compares the cbst of Algorithm 1 with the costs of the methods proposed in [ 1 ] and [ 4 ]. So about the cost Algorithm 1 is between other two methods.
394
J o u r n a l o f Shanghai University
Table 1
Costs of methods for solving the ILS problem
Algorithm 1
Algorithmin [1]
(S + A S l ) b = (A2 + AA2) V,
Algorithmin [4]
(4p + 4 q - 4 n / 3 ) n 2
or ( 2 p + 4 q + 2 n ) n z ( 5 p + 5 q - n ) n 2 ( 2 p + 2 q - 2 n / 3 ) n z p>>n
(2p + 4q)n z
(5p + 5q)n 2
(2p + 2q)n 2
p~n
(8n/3+4q)n z
(4n+Sq)n 2
(4n/3+2q)n z
where V is orthogonal, same as that in (5), II ~S~ II = O(u tl ~ II ), II aA2 II = O(u II A2 II ). In Step 3, the computed factor/~ in the hyperbolic factorization satisfies ( [4] ) [~:AR1
+AE1]
AS2]=Q[I 3
Error
Analysis
We will only consider the first order error bounds and ignore the possibility of overflow or underflow. We will use the letters with a hat for the matrices, vectors, or scalars computed in finite arithmetic. To show the backward stability we need the following two auxiliary results. L e m m a 2 Suppose D E R "× ~ is nonsingular upper bidiagonal and B E R q x 'L Let ~: be the numerical solution of the equation
(7)
0
(8)
,
where Q is orthogonal (which is related to H ), II ~R~ II , II AS2 II = O(u max{ II /~ II , I1 ~ II I),
II ZxE~ II = O(u). The computed vector )" satisfies y=[0
I,](cll+Adx)-(S+AS3)bz,
(9)
where II Ad~ II = 0 (u II da II = O (u II bl II ), II zxS3 II = O ( u II ~ II ) (See [8,pp. 76]). T h e v e c tots computed in Step 4 satisfy (/~ + A R 2 ) T w = f ,
(10)
(/~ + A R 3 ) z = W,
(11)
computed by forward substitution. Then ~: satisfies
(/~ + AD)y = z,
(12)
(f~ + A X ) D = B + AB,
where II AR2 II, II AR3 II ~ nu II /~ II , II A D II 3u II b II , s e e , e . g . , [ 8 , s e c . 8 . 1 ] . Finally the computed solution ~ satisfies
XD=B,
where II zxx II ~ 3 n u II ~ II, II AB II ~
