In this thesis, we study system identification and model reduction by using adaptive rational orthogonal basis (Takenaka-Malmquist (TM) system). The TM system has been studied since 1920s’ and is a general setting of rational orthogonal bases. The well-known Laguerre basis and Kautz basis are of its special cases. It has ample applications in both system identification and model reduction. In control theory, the purpose of system identification is to build mathematical models of dynamical systems based on measured data. It is very important because the controller design depends on the identified models. When the mathematical model of a system is complicated, it causes difficulty in both system analysis and controller design. Then model reduction is intended to find a simpler model, which matches some properties and aspects of the original model, to replace the original complex one in a given criterion. This thesis is arranged six parts. Chapter 1 is a background introduction. We briefly introduce the background of system identification using the rational orthogonal bases and give an introduction to model reduction problems. In chapter 2, we introduce the adaptive fourier decomposition (AFD) algorithm for Hardy-2 spaces. The AFD algorithm which is based on the TM system is to find an approximation by consecutively selecting the poles for the TM basis functions in the energy sense. In chapter 3, system identification using the AFD algorithm is presented. We introduce the two-step algorithm in this work. We modify the AFD algorithm for system identification and give some results on error estimations for different noise cases. After that, we study model reduction using TM system in chapter 4. We study the simultaneous selection of poles for the TM basis functions which leads to an algorithm of the best rational approximation. An extension work, backward shift algorithm, on rational functions is introduced in chapter 5. In the last chapter, some conclusions are ii given. It is noted that the theory and applications of AFD were proposed in the previous studies of Qian et al. The content of this thesis is a development of AFD in the area of control theory. Both of the research directions and techniques approaching to the topics in the individual sections are under close guidance of Prof Qian. The author also wishes to acknowledge his sincere thanks to Wan Feng and Michael Stessin for their assistance to this study.