## MB02ND

### Solution of Total Least-Squares problem using a partial SVD approach

[Specification] [Arguments] [Method] [References] [Comments] [Example]

Purpose

      To solve the Total Least Squares (TLS) problem using a Partial
      Singular Value Decomposition (PSVD) approach.
      The TLS problem assumes an overdetermined set of linear equations
      AX = B, where both the data matrix A as well as the observation
      matrix B are inaccurate. The routine also solves determined and
      underdetermined sets of equations by computing the minimum norm
      solution.
      It is assumed that all preprocessing measures (scaling, coordinate
      transformations, whitening, ... ) of the data have been performed
      in advance.

Specification

          SUBROUTINE MB02ND( M, N, L, RANK, THETA, C, LDC, X, LDX, Q, INUL,
         $                   TOL, RELTOL, IWORK, DWORK, LDWORK, BWORK,
         $                   IWARN, INFO )
    C     .. Scalar Arguments ..
          INTEGER           INFO, IWARN, L, LDC, LDWORK, LDX, M, N, RANK
          DOUBLE PRECISION  RELTOL, THETA, TOL
    C     .. Array Arguments ..
          LOGICAL           BWORK(*), INUL(*)
          INTEGER           IWORK(*)
          DOUBLE PRECISION  C(LDC,*), DWORK(*), Q(*), X(LDX,*)

Arguments

Input/Output Parameters

      M       (input) INTEGER
              The number of rows in the data matrix A and the
              observation matrix B.  M >= 0.

      N       (input) INTEGER
              The number of columns in the data matrix A.  N >= 0.

      L       (input) INTEGER
              The number of columns in the observation matrix B.
              L >= 0.

      RANK    (input/output) INTEGER
              On entry, if RANK < 0, then the rank of the TLS
              approximation [A+DA|B+DB] (r say) is computed by the
              routine.
              Otherwise, RANK must specify the value of r.
              RANK <= min(M,N).
              On exit, if RANK < 0 on entry and INFO = 0, then RANK
              contains the computed rank of the TLS approximation
              [A+DA|B+DB].
              Otherwise, the user-supplied value of RANK may be
              changed by the routine on exit if the RANK-th and the
              (RANK+1)-th singular values of C = [A|B] are considered
              to be equal, or if the upper triangular matrix F (as
              defined in METHOD) is (numerically) singular.

      THETA   (input/output) DOUBLE PRECISION
              On entry, if RANK < 0, then the rank of the TLS
              approximation [A+DA|B+DB] is computed using THETA as
              (min(M,N+L) - d), where d is the number of singular
              values of [A|B] <= THETA. THETA >= 0.0.
              Otherwise, THETA is an initial estimate (t say) for
              computing a lower bound on the RANK largest singular
              values of [A|B]. If THETA < 0.0 on entry however, then
              t is computed by the routine.
              On exit, if RANK >= 0 on entry, then THETA contains the
              computed bound such that precisely RANK singular values
              of C = [A|B] are greater than THETA + TOL.
              Otherwise, THETA is unchanged.

      C       (input/output) DOUBLE PRECISION array, dimension (LDC,N+L)
              On entry, the leading M-by-(N+L) part of this array must
              contain the matrices A and B. Specifically, the first N
              columns must contain the data matrix A and the last L
              columns the observation matrix B (right-hand sides).
              On exit, if INFO = 0, the first N+L components of the
              columns of this array whose index i corresponds with
              INUL(i) = .TRUE., are the possibly transformed (N+L-RANK)
              base vectors of the right singular subspace corresponding
              to the singular values of C = [A|B] which are less than or
              equal to THETA. Specifically, if L = 0, or if RANK = 0 and
              IWARN <> 2, these vectors are indeed the base vectors
              above. Otherwise, these vectors form the matrix V2,
              transformed as described in Step 4 of the PTLS algorithm
              (see METHOD). The TLS solution is computed from these
              vectors. The other columns of array C contain no useful
              information.

      LDC     INTEGER
              The leading dimension of array C.  LDC >= max(1,M,N+L).

      X       (output) DOUBLE PRECISION array, dimension (LDX,L)
              If INFO = 0, the leading N-by-L part of this array
              contains the solution X to the TLS problem specified by
              A and B.

      LDX     INTEGER
              The leading dimension of array X.  LDX >= max(1,N).

      Q       (output) DOUBLE PRECISION array, dimension
              (max(1,2*min(M,N+L)-1))
              This array contains the partially diagonalized bidiagonal
              matrix J computed from C, at the moment that the desired
              singular subspace has been found. Specifically, the
              leading p = min(M,N+L) entries of Q contain the diagonal
              elements q(1),q(2),...,q(p) and the entries Q(p+1),Q(p+2),
              ...,Q(2*p-1) contain the superdiagonal elements e(1),e(2),
              ...,e(p-1) of J.

      INUL    (output) LOGICAL array, dimension (N+L)
              The indices of the elements of this array with value
              .TRUE. indicate the columns in C containing the base
              vectors of the right singular subspace of C from which
              the TLS solution has been computed.

Tolerances

      TOL     DOUBLE PRECISION
              This parameter defines the multiplicity of singular values
              by considering all singular values within an interval of
              length TOL as coinciding. TOL is used in checking how many
              singular values are less than or equal to THETA. Also in
              computing an appropriate upper bound THETA by a bisection
              method, TOL is used as a stopping criterion defining the
              minimum (absolute) subinterval width. TOL is also taken
              as an absolute tolerance for negligible elements in the
              QR/QL iterations. If the user sets TOL to be less than or
              equal to 0, then the tolerance is taken as specified in
              SLICOT Library routine MB04YD document.

      RELTOL  DOUBLE PRECISION
              This parameter specifies the minimum relative width of an
              interval. When an interval is narrower than TOL, or than
              RELTOL times the larger (in magnitude) endpoint, then it
              is considered to be sufficiently small and bisection has
              converged. If the user sets RELTOL to be less than
              BASE * EPS, where BASE is machine radix and EPS is machine
              precision (see LAPACK Library routine DLAMCH), then the
              tolerance is taken as BASE * EPS.

Workspace

      IWORK   INTEGER array, dimension (N+2*L)

      DWORK   DOUBLE PRECISION array, dimension (LDWORK)
              On exit, if INFO = 0, DWORK(1) returns the optimal value
              of LDWORK, and DWORK(2) returns the reciprocal of the
              condition number of the matrix F.

      LDWORK  INTEGER
              The length of the array DWORK.
              LDWORK = max(2, max(M,N+L) + 2*min(M,N+L),
                           min(M,N+L) + LW + max(6*(N+L)-5,
                                                 L*L+max(N+L,3*L)),
              where
              LW = (N+L)*(N+L-1)/2,  if M >= N+L,
              LW = M*(N+L-(M-1)/2),  if M <  N+L.
              For optimum performance LDWORK should be larger.

              If LDWORK = -1, then a workspace query is assumed;
              the routine only calculates the optimal size of the
              DWORK array, returns this value as the first entry of
              the DWORK array, and no error message related to LDWORK
              is issued by XERBLA.

      BWORK   LOGICAL array, dimension (N+L)

Warning Indicator

      IWARN   INTEGER
              = 0:  no warnings;
              = 1:  if the rank of matrix C has been lowered because a
                    singular value of multiplicity greater than 1 was
                    found;
              = 2:  if the rank of matrix C has been lowered because the
                    upper triangular matrix F is (numerically) singular.

Error Indicator

      INFO    INTEGER
              = 0:  successful exit;
              < 0:  if INFO = -i, the i-th argument had an illegal
                    value;
              = 1:  if the maximum number of QR/QL iteration steps
                    (30*MIN(M,N)) has been exceeded;
              = 2:  if the computed rank of the TLS approximation
                    [A+DA|B+DB] exceeds MIN(M,N). Try increasing the
                    value of THETA or set the value of RANK to min(M,N).

Method

      The method used is the Partial Total Least Squares (PTLS) approach
      proposed by Van Huffel and Vandewalle [5].

      Let C = [A|B] denote the matrix formed by adjoining the columns of
      B to the columns of A on the right.

      Total Least Squares (TLS) definition:
      -------------------------------------

        Given matrices A and B, find a matrix X satisfying

             (A + DA) X = B + DB,

        where A and DA are M-by-N matrices, B and DB are M-by-L matrices
        and X is an N-by-L matrix.
        The solution X must be such that the Frobenius norm of [DA|DB]
        is a minimum and each column of B + DB is in the range of
        A + DA. Whenever the solution is not unique, the routine singles
        out the minimum norm solution X.

      Let V denote the right singular subspace of C. Since the TLS
      solution can be computed from any orthogonal basis of the subspace
      of V corresponding to the smallest singular values of C, the
      Partial Singular Value Decomposition (PSVD) can be used instead of
      the classical SVD. The dimension of this subspace of V may be
      determined by the rank of C or by an upper bound for those
      smallest singular values.

      The PTLS algorithm proceeds as follows (see [2 - 5]):

      Step 1: Bidiagonalization phase
              -----------------------
       (a) If M is large enough than N + L, transform C into upper
           triangular form R by Householder transformations.
       (b) Transform C (or R) into upper bidiagonal form
           (p = min(M,N+L)):

                      |q(1) e(1)  0   ...  0   |
                 (0)  | 0   q(2) e(2)      .   |
                J   = | .                  .   |
                      | .                e(p-1)|
                      | 0             ... q(p) |

           if M >= N + L, or lower bidiagonal form:

                      |q(1)  0    0   ...  0     0   |
                 (0)  |e(1) q(2)  0        .     .   |
                J   = | .                  .     .   |
                      | .                 q(p)   .   |
                      | 0             ... e(p-1) q(p)|

           if M < N + L, using Householder transformations.
           In the second case, transform the matrix to the upper
           bidiagonal form by applying Givens rotations.
       (c) Initialize the right singular base matrix with the identity
           matrix.

      Step 2: Partial diagonalization phase
              -----------------------------
      If the upper bound THETA is not given, then compute THETA such
      that precisely p - RANK singular values (p=min(M,N+L)) of the
      bidiagonal matrix are less than or equal to THETA, using a
      bisection method [5]. Diagonalize the given bidiagonal matrix J
      partially, using either QL iterations (if the upper left diagonal
      element of the considered bidiagonal submatrix is smaller than the
      lower right diagonal element) or QR iterations, such that J is
      split into unreduced bidiagonal submatrices whose singular values
      are either all larger than THETA or are all less than or equal
      to THETA. Accumulate the Givens rotations in V.

      Step 3: Back transformation phase
              -------------------------
      Apply the Householder transformations of Step 1(b) onto the base
      vectors of V associated with the bidiagonal submatrices with all
      singular values less than or equal to THETA.

      Step 4: Computation of F and Y
              ----------------------
      Let V2 be the matrix of the columns of V corresponding to the
      (N + L - RANK) smallest singular values of C.
      Compute with Householder transformations the matrices F and Y
      such that:

                        |VH   Y|
               V2 x Q = |      |
                        |0    F|

      where Q is an orthogonal matrix, VH is an N-by-(N-RANK) matrix,
      Y is an N-by-L matrix and F is an L-by-L upper triangular matrix.
      If F is singular, then reduce the value of RANK by one and repeat
      Steps 2, 3 and 4.

      Step 5: Computation of the TLS solution
              -------------------------------
      If F is non-singular then the solution X is obtained by solving
      the following equations by forward elimination:

               X F = -Y.

      Notes:
      If RANK is lowered in Step 4, some additional base vectors must
      be computed in Step 2. The additional computations are kept to
      a minimum.
      If RANK is lowered in Step 4 but the multiplicity of the RANK-th
      singular value is larger than 1, then the value of RANK is further
      lowered with its multiplicity defined by the parameter TOL. This
      is done at the beginning of Step 2 by calling SLICOT Library
      routine MB03MD (from MB04YD), which estimates THETA using a
      bisection method. If F in Step 4 is singular, then the computed
      solution is infinite and hence does not satisfy the second TLS
      criterion (see TLS definition). For these cases, Golub and
      Van Loan [1] claim that the TLS problem has no solution. The
      properties of these so-called nongeneric problems are described
      in [6] and the TLS computations are generalized in order to solve
      them. As proven in [6], the proposed generalization satisfies the
      TLS criteria for any number L of observation vectors in B provided
      that, in addition, the solution | X| is constrained to be
                                      |-I|
      orthogonal to all vectors of the form |w| which belong to the
                                            |0|
      space generated by the columns of the submatrix |Y|.
                                                      |F|

References

      [1] Golub, G.H. and Van Loan, C.F.
          An Analysis of the Total Least-Squares Problem.
          SIAM J. Numer. Anal., 17, pp. 883-893, 1980.

      [2] Van Huffel, S., Vandewalle, J. and Haegemans, A.
          An Efficient and Reliable Algorithm for Computing the
          Singular Subspace of a Matrix Associated with its Smallest
          Singular Values.
          J. Comput. and Appl. Math., 19, pp. 313-330, 1987.

      [3] Van Huffel, S.
          Analysis of the Total Least Squares Problem and its Use in
          Parameter Estimation.
          Doctoral dissertation, Dept. of Electr. Eng., Katholieke
          Universiteit Leuven, Belgium, June 1987.

      [4] Chan, T.F.
          An Improved Algorithm for Computing the Singular Value
          Decomposition.
          ACM TOMS, 8, pp. 72-83, 1982.

      [5] Van Huffel, S. and Vandewalle, J.
          The Partial Total Least Squares Algorithm.
          J. Comput. Appl. Math., 21, pp. 333-341, 1988.

      [6] Van Huffel, S. and Vandewalle, J.
          Analysis and Solution of the Nongeneric Total Least Squares
          Problem.
          SIAM J. Matr. Anal. and Appl., 9, pp. 360-372, 1988.

Numerical Aspects

      The computational efficiency of the PTLS algorithm compared with
      the classical TLS algorithm (see [2 - 5]) is obtained by making
      use of PSVD (see [1]) instead of performing the entire SVD.
      Depending on the gap between the RANK-th and the (RANK+1)-th
      singular values of C, the number (N + L - RANK) of base vectors to
      be computed with respect to the column dimension (N + L) of C and
      the desired accuracy RELTOL, the algorithm used by this routine is
      approximately twice as fast as the classical TLS algorithm at the
      expense of extra storage requirements, namely:
        (N + L) x (N + L - 1)/2  if M >= N + L or
        M x (N + L - (M - 1)/2)  if M <  N + L.
      This is because the Householder transformations performed on the
      rows of C in the bidiagonalization phase (see Step 1) must be kept
      until the end (Step 5).

Further Comments

      None

Example

Program Text

    *     MB02ND EXAMPLE PROGRAM TEXT
    *
    *     .. Parameters ..
          DOUBLE PRECISION ZERO
          PARAMETER        ( ZERO = 0.0D0 )
          INTEGER          NIN, NOUT
          PARAMETER        ( NIN = 5, NOUT = 6 )
          INTEGER          MMAX, NMAX, LMAX
          PARAMETER        ( MMAX = 20, NMAX = 20, LMAX = 20 )
          INTEGER          LDC, LDX
          PARAMETER        ( LDC = MAX( MMAX, NMAX+LMAX ), LDX = NMAX )
          INTEGER          LENGQ
          PARAMETER        ( LENGQ = 2*MIN(MMAX,NMAX+LMAX)-1 )
          INTEGER          LIWORK
          PARAMETER        ( LIWORK = NMAX+2*LMAX )
          INTEGER          LDWORK
          PARAMETER        ( LDWORK = MAX(2, MAX( MMAX, NMAX+LMAX ) +
         $                            2*MIN( MMAX, NMAX+LMAX ),
         $                            MIN( MMAX, NMAX+LMAX ) +
         $                            MAX( ( NMAX+LMAX )*( NMAX+LMAX-1 )/2,
         $                              MMAX*( NMAX+LMAX-( MMAX-1 )/2 ) ) +
         $                            MAX( 6*(NMAX+LMAX)-5, LMAX*LMAX +
         $                                 MAX( NMAX+LMAX, 3*LMAX ) ) ) )
          INTEGER          LBWORK
          PARAMETER        ( LBWORK = NMAX+LMAX )
    *     .. Local Scalars ..
          DOUBLE PRECISION RELTOL, THETA, THETA1, TOL
          INTEGER          I, INFO, IWARN, J, K, L, LOOP, M, MINMNL, N,
         $                 RANK, RANK1
    *     .. Local Arrays ..
          DOUBLE PRECISION C(LDC,NMAX+LMAX), DWORK(LDWORK),
         $                 Q(LENGQ), X(LDX,LMAX)
          INTEGER          IWORK(LIWORK)
          LOGICAL          BWORK(LBWORK), INUL(NMAX+LMAX)
    *     .. External Subroutines ..
          EXTERNAL         MB02ND
    *     .. Intrinsic Functions ..
          INTRINSIC        MAX, MIN
    *     .. Executable Statements ..
    *
          WRITE ( NOUT, FMT = 99999 )
    *     Skip the heading in the data file and read the data.
          READ ( NIN, FMT = '()' )
          READ ( NIN, FMT = * ) M, N, L, RANK, THETA, TOL, RELTOL
          IF ( M.LT.0 .OR. M.GT.MMAX ) THEN
             WRITE ( NOUT, FMT = 99982 ) M
          ELSE IF ( N.LT.0 .OR. N.GT.NMAX ) THEN
             WRITE ( NOUT, FMT = 99983 ) N
          ELSE IF ( L.LT.0 .OR. L.GT.LMAX ) THEN
             WRITE ( NOUT, FMT = 99981 ) L
          ELSE IF ( RANK.GT.MIN( MMAX, NMAX ) ) THEN
             WRITE ( NOUT, FMT = 99980 ) RANK
          ELSE IF ( RANK.LT.0 .AND. THETA.LT.ZERO ) THEN
             WRITE ( NOUT, FMT = 99979 ) THETA
          ELSE
             READ ( NIN, FMT = * ) ( ( C(I,J), J = 1,N+L ), I = 1,M )
             RANK1 = RANK
             THETA1 = THETA
    *        Compute the solution to the TLS problem Ax = b.
             CALL MB02ND( M, N, L, RANK, THETA, C, LDC, X, LDX, Q, INUL,
         $                TOL, RELTOL, IWORK, DWORK, LDWORK, BWORK, IWARN,
         $                INFO )
    *
             IF ( INFO.NE.0 ) THEN
                WRITE ( NOUT, FMT = 99998 ) INFO
             ELSE
                IF ( IWARN.NE.0 ) THEN
                   WRITE ( NOUT, FMT = 99997 ) IWARN
                   WRITE ( NOUT, FMT = 99996 ) RANK
                ELSE
                   IF ( RANK1.LT.0 ) WRITE ( NOUT, FMT = 99996 ) RANK
                END IF
                IF ( THETA1.LT.ZERO ) WRITE ( NOUT, FMT = 99995 ) THETA
                WRITE ( NOUT, FMT = 99994 )
                MINMNL = MIN( M, N+L )
                LOOP = MINMNL - 1
                DO 20 I = 1, LOOP
                   K = I + MINMNL
                   WRITE ( NOUT, FMT = 99993 ) I, I, Q(I), I, I + 1, Q(K)
       20       CONTINUE
                WRITE ( NOUT, FMT = 99992 ) MINMNL, MINMNL, Q(MINMNL)
                WRITE ( NOUT, FMT = 99991 )
                DO 60 J = 1, L
                   DO 40 I = 1, N
                      WRITE ( NOUT, FMT = 99990 ) X(I,J)
       40          CONTINUE
                   IF ( J.LT.L ) WRITE ( NOUT, FMT = 99989 )
       60       CONTINUE
                WRITE ( NOUT, FMT = 99987 ) N + L, N + L
                WRITE ( NOUT, FMT = 99985 )
                DO 80 I = 1, MAX( M, N + L )
                   WRITE ( NOUT, FMT = 99984 ) ( C(I,J), J = 1,N+L )
       80       CONTINUE
                WRITE ( NOUT, FMT = 99986 )
                DO 100 J = 1, N + L
                   WRITE ( NOUT, FMT = 99988 ) J, INUL(J)
      100       CONTINUE
             END IF
          END IF
          STOP
    *
    99999 FORMAT (' MB02ND EXAMPLE PROGRAM RESULTS',/1X)
    99998 FORMAT (' INFO on exit from MB02ND = ',I2)
    99997 FORMAT (' IWARN on exit from MB02ND = ',I2,/)
    99996 FORMAT (' The computed rank of the TLS approximation  = ',I3,/)
    99995 FORMAT (' The computed value of THETA = ',F7.4,/)
    99994 FORMAT (' The elements of the partially diagonalized bidiagonal ',
         $       'matrix are',/)
    99993 FORMAT (2(' (',I1,',',I1,') = ',F7.4,2X))
    99992 FORMAT (' (',I1,',',I1,') = ',F7.4,/)
    99991 FORMAT (' The solution X to the TLS problem is ',/)
    99990 FORMAT (1X,F8.4)
    99989 FORMAT (' ')
    99988 FORMAT (I3,L8)
    99987 FORMAT (/' Right singular subspace corresponds to the first ',I2,
         $       ' components of the j-th ',/' column of C for which INUL(',
         $       'j) = .TRUE., j = 1,...,',I2,/)
    99986 FORMAT (/'  j    INUL(j)',/)
    99985 FORMAT (' Matrix C',/)
    99984 FORMAT (20(1X,F8.4))
    99983 FORMAT (/' N is out of range.',/' N = ',I5)
    99982 FORMAT (/' M is out of range.',/' M = ',I5)
    99981 FORMAT (/' L is out of range.',/' L = ',I5)
    99980 FORMAT (/' RANK is out of range.',/' RANK = ',I5)
    99979 FORMAT (/' THETA must be at least zero.',/' THETA = ',F8.4)
          END

Program Data

     MB02ND EXAMPLE PROGRAM DATA
       6     3     1     -1     0.001     0.0     0.0
       0.80010  0.39985  0.60005  0.89999
       0.29996  0.69990  0.39997  0.82997
       0.49994  0.60003  0.20012  0.79011
       0.90013  0.20016  0.79995  0.85002
       0.39998  0.80006  0.49985  0.99016
       0.20002  0.90007  0.70009  1.02994

Program Results

     MB02ND EXAMPLE PROGRAM RESULTS

     The computed rank of the TLS approximation  =   3

     The elements of the partially diagonalized bidiagonal matrix are

     (1,1) =  3.2280   (1,2) = -0.0287
     (2,2) =  0.8714   (2,3) =  0.0168
     (3,3) =  0.3698   (3,4) =  0.0000
     (4,4) =  0.0001

     The solution X to the TLS problem is

       0.5003
       0.8003
       0.2995

     Right singular subspace corresponds to the first  4 components of the j-th
     column of C for which INUL(j) = .TRUE., j = 1,..., 4

     Matrix C

      -0.3967  -0.7096   0.4612  -0.3555
       0.9150  -0.2557   0.2414  -0.5687
      -0.0728   0.6526   0.5215  -0.2128
       0.0000   0.0720   0.6761   0.7106
       0.1809   0.3209   0.0247  -0.4139
       0.0905   0.4609  -0.3528   0.5128

      j    INUL(j)

      1       F
      2       F
      3       F
      4       T

Return to index