Artificial Neural Network which means multilayer feedforward ANN (Artificial Neural Network) here is only a system loosely modeled on the human brain and many applications of ANN have been found growing in many different disciplines. In this thesis, we start with introducing the historical background and definition of ANN. Also, we will build up an indexing system for the parameters and variables of ANN. The indexing system is consistent with matrix operations. So we are able to avoid unnecessary complexity when we implement or analyse this kind of ANN. Moreover, the error function has been investigated statistically. Minimizing the sum of squares type error function, the model output essentially approaches to the conditional expectation of the target vector when the input vector is given. A local theorem of steepest descent algorithm which is a classical optimization method is offered in this thesis. We are able to observe that the key of the steepest descent algorithm is getting the useful information of optimization from the derivatives of the objective function. It leads to the discussion for ANN. Backpropagation algorithm is one of the complicated algorithms around. For simplicity, we have performed BP (Backpropagation) for a simple but typical ANN first so that we are able to understand the relation among all the parameters and variables within the network as well as the key idea how Backpropagation works. Actually, BP provides us with a method for calculating the gradient of the error function of an ANN model. For the general case, we put it in theorem forms and prove them in very detail. Lastly, an ANN simulator and BP have been implemented on the Matlab platform. Some technical remarks for implementation have been given. The program is tested by a mathematical chaotic time series. The predictor actually is able to capture the data generator precisely in this case. A real world time series of SO₂ (Sulphur dioxide) concentration has also been predicted. We have 8 days out of 18 days which error is within 14%. So ANN can be considered as a useful tool for time series prediction which is complementary to conventional methods.