Multi-agent systems are becoming increasingly important in our society because they can be used to naturally solve a wide variety of problem; the thesis addresses multi-agent systems based on the robot soccer system- the good test bed for studying multi-agent systems. Robot soccer system operates in a real time, complex, dynamic, unpredictable environment and real time control strategy is a crucial part of it, the goal of the thesis is to give a general approach to solve the real time control strategy problem, so we define an approach of building real time control strategy through the following step, firstly, we define a three layer strategic architecture within which a flexible team structure is presented, allowing agents to decompose the task space into flexible roles, actions and behavior, and allowing them to smoothly switch roles while acting. Second, we use AUML as a specification tool to design the architecture. Third, the thesis introduces the approaches to implement the low level and high-level functions. And implementing the approach to the 3v3 robot soccer system demonstrates the effectiveness of it. Keywords: Multi-agent system, Robot Soccer System, Reinforcement Learning