Abstract
Circulant matrices have been extensively applied in Symbolic and Numerical Computations, but we study their new application, namely, to randomized pre-processing that supports Gaussian elimination with no pivoting, hereafter referred to as GENP. We prove that, with a probability close to 1, GENP proceeds with no divisions by 0 if the input matrix is pre-processed with a random circulant multiplier. This yields 4-fold acceleration (in the cases of both general and structured input matrices) versus pre-processing with the pair of random triangular Toeplitz multipliers, which has been the user’s favorite since 1991. In that part of our paper, we assume computations with infinite precision, but in other parts with double precision, in the presence of rounding errors. In this case, GENP fails without pre-processing unless all square leading blocks of the input matrix are well-conditioned, but empirically GENP produces accurate output consistently if a well-conditioned input matrix is pre-processed with random circulant multipliers. We also support formally the latter empirical observation if we allow standard Gaussian random input and hence the average non-singular and well-conditioned input as well, but we prove that GENP fails numerically with a probability close to 1 in the case of some specific input matrix pre-processed with such multipliers. We also prove that even for the worst case well-conditioned input, GENP runs into numerical problems only with a probability close to 0, if a nonsingular and well-conditioned input matrix is multiplied by a standard Gaussian random matrix. All our results for GENP can be readily extended to the highly important block Gaussian elimination.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bunch, J.R.: Stability of methods for solving Toeplitz systems of equations. SIAM J. Sci. and Stat. Computing 6(2), 349–364 (1985)
Bini, D., Pan, V.Y.: Polynomial and Matrix Computations, Fundamental Algorithms, vol. 1. Birkhäuser, Boston (1994)
Bitmead, R.R., Anderson, B.D.O.: Asymptotically fast solution of Toeplitz and related systems of linear equations. Linear Algebra and Its Applications 34, 103–116 (1980)
Ballard, G., Carson, E., Demmel, J., Hoemmen, M., Knight, N., Schwartz, O.: Communication lower bounds and optimal algorithms for numerical linear algebra. Acta Numerica 23, 1–155 (2014)
Chen, Z., Dongarra, J.J.: Condition numbers of Gaussian random matrices. SIAM. J. Matrix Analysis and Applications 27, 603–620 (2005)
Chen, L., Eberly, W., Kaltofen, E., Saunders, B.D., Turner, W.J., Villard, G.: Efficient matrix preconditioners for black box linear algebra. Linear Algebra and Its Applications 343–344, 119–146 (2002)
Chan, R.H., Ng, M.K.: Conjugate gradient methods for Toeplitz systems. SIAM Review 38, 427–482 (1996)
Chan, R.H., Ng, M.K.: Iterative methods for linear systems with matrix structures. In: Kailath, T., Sayed, A.H. (eds.) Fast Reliable Algorithms for Matrices with Structure, pp. 117–152. SIAM, Philadelpha (1999)
Cline, R.E., Plemmons, R.J., Worm, G.: Generalized inverses of certain Toeplitz matrices. Linear Algebra and Its Applications 8, 25–33 (1974)
Demmel, J.: The probability that a numerical analysis problem is difficult. Math. Comput. 50, 449–480 (1988)
Demillo, R.A., Lipton, R.J.: A probabilistic remark on algebraic program testing. Information Processing Letters 7(4), 193–195 (1978)
Davidson, K.R., Szarek, S.J.: Local operator theory, random matrices, and Banach spaces. In: Johnson, W.B., Lindenstrauss, J. (eds.) Handbook on the Geometry of Banach Spaces, pp. 317–368. North Holland, Amsterdam (2001)
Edelman, A.: Eigenvalues and condition numbers of random matrices. SIAM J. Matrix Analysis and Applications 9(4), 543–560 (1988)
Edelman, A., Sutton, B.D.: Tails of condition number distributions. SIAM J. Matrix Anal. and Applications 27(2), 547–560 (2005)
Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. Johns Hopkins University Press, Baltimore (2013)
Halko, N., Martinsson, P.G., Tropp, J.A.: Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review 53(2), 217–288 (2011)
Kaltofen, E., Saunders, B.D.: On Wiedemann’s method for solving sparse linear systems. In: Mattson, H.F., Mora, T., Rao, T.R.N. (eds.) AAECC 1991. LNCS, vol. 539, pp. 29–38. Springer, Heidelberg (1991)
Morf, M.: Fast Algorithms for Multivariable Systems. Ph.D. Thesis, Department of Electrical Engineering, Stanford University, Stanford, CA (1974)
Morf, M.: Doubling algorithms for toeplitz and related equations. In: Proc. IEEE International Conference on ASSP, pp. 954–959. IEEE Press, Piscataway (1980)
Pan, V.Y.: Structured Matrices and Polynomials: Unified Superfast Algorithms. Birkhäuser/Springer, Boston/New York (2001)
Pan, V.Y.: How Bad Are Vandermonde Matrices? Available at arxiv: 1504.02118, April 8, 2015 (revised April 26, 2015 and June 2015)
Pan, V.Y.: Transformations of matrix structures work again. Linear Algebra and Its Applications 465, 1–32 (2015)
Pan, V.Y., Qian, G., Yan, X.: Random multipliers numerically stabilize Gaussian and block Gaussian elimination: Proofs and an extension to low-rank approximation. Linear Algebra and Its Applications 481, 202–234 (2015)
Pan, V.Y., Qian, G., Zheng, A.: Randomized preprocessing versus pivoting. Linear Algebra and Its Applications 438(4), 1883–1899 (2013)
Pan, V.Y., Svadlenka, J., Zhao, L.: Estimating the norms of circulant and Toeplitz random matrices and their inverses. Linear Algebra and Its Applications 468, 197–210 (2015)
Pan, V.Y., Wang, X.: Degeneration of integer matrices modulo an integer. Linear Algebra and Its Applications 429, 2113–2130 (2008)
Pan, V.Y., Zhao, L.: Gaussian Elimination with Randomized Pre-processing, arxiv, 1501.05385, June 15, 2015
Schwartz, J.T.: Fast probabilistic algorithms for verification of polynomial identities. J. ACM 27(4), 701–717 (1980)
Zippel, R.E.: Probabilistic algorithms for sparse polynomials. In: Ng, E.W. (ed.) EUROSAM 1979. LNCS, vol. 72, pp. 216–226. Springer, Berlin (1979)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Pan, V.Y., Zhao, L. (2015). Randomized Circulant and Gaussian Pre-processing. In: Gerdt, V., Koepf, W., Seiler, W., Vorozhtsov, E. (eds) Computer Algebra in Scientific Computing. CASC 2015. Lecture Notes in Computer Science(), vol 9301. Springer, Cham. https://doi.org/10.1007/978-3-319-24021-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-24021-3_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24020-6
Online ISBN: 978-3-319-24021-3
eBook Packages: Computer ScienceComputer Science (R0)