Fractional differential equations have recently received much attention within computational mathematics and applied science. Their numerical treatment is an important research area as such equations pose substantial challenges to existing algorithms. In the first part of this thesis, we review some basic concepts and introduce some approximate methods of the fractional derivatives. The computational challenge of the time-fractional differential equations are also discussed. The main research topic of this thesis is the fast solver for solving time fractional differential equations. Finite difference methods are one of the most important classes of numerical methods for solving fractional partial differential equations. In the second part of this thesis, we review the most existing classical and well-studied finite difference methods for the time fractional diffusion equations. To reduce the computation cost and storage, the solution is sought in the structured matrices. We carefully explore the structure of the coefficient matrices, and then we find that the discretization of this equations typically involves matrices with a block triangular Toeplitz structure. In the third part of this thesis, we consider a block lower triangular Toeplitz with tridiagonal blocks (BL3TB) system which arises from the finite difference discretization of time fractional partial differential equations. The BL3TB matrix is approximated by a block ϵ-circulant matrix, which can be efficiently inverted using the fast Fourier transforms. The error estimation is also carefully discussed to show the high accuracy of the approximation by our fast algorithm. The fourth part is devoted to verifying that all the classical finite difference schemes presented earlier are suitable for the sufficient conditions of our algorithm. Some numerical tests are reported to illustrate the efficiency of the proposed method for solving the time fractional sub-diffusion equations. Finally, conclusion and possible research topics in our further study are given at the end of this thesis.