Kalman filter is one of the most well known Bayesian state estimation algorithm, which was originally developed by Kalman and Bucy for predicting and filtering random signals of linear systems with Gaussian uncertainties. Later, Kalman filter was modified to give the extended Kalman filter to accommodate lightly nonlinear systems. From then on, they have found widespread applications throughout all branches of science and engineering and also have become a popular tool in structural identification. However, the statistics (or covariance) of process noise and measurement noise of Kalman filter are assumed to be known, but this assumption is difficult to fulfill in practice. Choosing different measurement noise covariance has been shown to affect the performance of Kalman filter. Therefore, various adaptive filters have already been proposed to estimate the noise covariance. Nevertheless, one of the biggest common properties shared by these various adaptive filters is that the estimation of noise covariance is based on a batch number of previous measurements (sometime it is called the filtering window), in other words, these adaptive filters are not able to estimate and update noise covariance in actually real time, especially for the case where the noise parameters are time variant, which makes them unattractive and limitedly useful in application. In this paper, a novel algorithm is proposed to online estimate the noise covariance under the assumption that the system parameters are perfectly known. First, both the noise covariance are expressed as the product of time variant factors (noise parameters) and normalizing matrixes (independent with time), then Gaussian distribution is utilized to describe the prior information of the noise parameters, and finally the posterior PDF of noise parameters can be directly obtained based on Bayesian theory when the new measurements are available. Herein, the Gaussian distribution is used again to approximate the posterior PDF and is subsequently treated as the prior of the next step. So the noise parameters can be estimated and updated at every step by maximizing the posterior PDF. Meanwhile, the associated uncertainties are also available by computing the Hessian matrix. In addition, the fading factor is brought in to enhance the performance of the proposed algorithm. Compared with the previous adaptive filters, our proposed algorithm has the following advantages: 1) estimate the noise parameters in real time; 2) not only provide the estimation of noise parameters but also their uncertainties; 3) the previous measurements are not necessary to be stored; 4) highly computational efficiency since no iteration is required; 5) quite applicable in the situation where the noise parameters are time variant. The proposed algorithm is applied on two examples using a fifty-story building, subjected to ground motion and wind loading, respectively. It is demonstrated that the proposed algorithm is successful and with high superiority.