In this thesis, we apply Bayesian perspective to solve linear systems, and we first survey various numerical schemes and methods for large sized linear problems. Generally speaking, when the prior and likelihood density function are Gaussian, then the posterior density function is also Gaussian. But for a large sized and ill-conditioned inverse problems, we can hardly treat the posterior covariance matrix, or calculate the solutions of the problems even if we have the explicit formulae. Therefore, we improve the schemes and make some new proposals for general matrix problems, especially for matrix that is dense with large condition number, or even hasn’t any special structure. We employ iterative formulae and prior-preconditioned Hessian of the data misfit, because the eigenvalues of Hessian matrix decrease rapidly to zero for most of the ill-posed problems, then we can take a low-rank approximation to reduce the operation cost. The analysis of convergence of our proposals are considered in between, and some of the factors that affect the convergence rate are given out through theoretical proofs. We realize the algorithms through numerical experiments based on MATLAB programs, and different sets of parameter values are tested as well. 4 The thesis is divided into five chapters. A brief introduction to classic numerical methods: direct methods and iterative methods, as well as two recent perspectives in solving linear systems: Monte Carlo perspective and Bayesian perspective. Some approaches in statistics are also reviewed in Chapter 1. In Chapter 2, a brief revision of Bayes’ theorem is given early in this chapter, continuing with the analysis of the Gaussian case, and the theoretical proofs of the resultant posterior distribution are provided. An introduction to Monte Carlo methods is given as well in the end of the chapter. In Chapter 3, we consider two useful estimators, i.e. Conditional Mean Estimator and Maximum A Posteriori Estimator, of finding an approximation of a problem. We first start by numerical schemes under Markov Chain Monte Carlo perspective (Metropolis-Hasting, Componentwise MetropolisHasting, Gibbs sampling scheme), and continue with the preconditioned iterative method from Bayesian inversion perspective in the latter part of the chapter. In addition, we realize the corresponding algorithms and numerical experiments to find out the relations among the parameters of favor. Being the main part of this thesis, three new algorithms for solving linear systems in Bayesian perspective are proposed in Chapter 4. The idea of proposing these methods is to provide a way for solving large sized and ill-conditioned linear systems. Iterative formulae and low-rank approximation are employed in our scheme, and spectral decomposition of the priorpreconditioned Hessian of the data misfit is taken into account because the eigenvalues of the Hessian matrix decrease rapidly to zero if the matrix is ill-conditioned. By appropriately selecting the parameter values used in the algorithms, a good approximation can be obtained within a few number of iteration, compared with that of finding the inverse of a matrix, and solve the linear problems using direct methods, which is prohibited when the size of the matrix is so large. Also, analytical proofs of the iterative formulae and the approximation used are given and numerical experiments are done to demonstrate their consistency. We end up with a summary of the experiments done in this thesis, and a comparison among different parameter values is discussed as well, continuing with a conclusion of this thesis in Chapter 5.