As one of the most cutting-edge, machine-learning techniques, deep learning caves out a research wave in machine learning and makes significant progress in artificial intelligence. It is a major step forward to high-level intelligence, and is ranked first in the MIT technology review list of top-10 breakthroughs of 2013 1 . Deep learning is about learning multiple levels of representation and abstraction that help to make sense of data such as images, sound, and text. The early deep-learning papers have received thousands of citations in the past decade, while they significantly promoted the development of artificial intelligence in both academic research and industrial applications. The dissertation develops and investigates new sophisticated deep learning techniques and their extension applications in pattern recognition, including image recognition, image inpainting and time-series prediction. Although deep learning has made huge success in pattern recognition, image recognition, speech recognition and video processing in the past decade, it is still in its infant phase of development. There are still many open issues of deep learning remained to settle. First, the high complexity of current deep learning algorithms remains a tough problem. For a medium-large deep model, it may take several days or even weeks to fine-tune it. It is unacceptable for some applications that require real-time learning and response. A distributed learning algorithm, which is based on the big data platform Hadoop, has been proposed to alleviate the problem. Second, deep learning was initially designed for image recognition. Based on the main idea of deep learning, we has developed a predictive deep model, called predictive deep Boltzmann machine, to analyse highly varying and super unsmooth time series. Wind speed is difficult to predict due to its volatility and deviation. It is desirable to evaluate the performance of the proposed predictive deep Boltzmann machine in wind speed prediction. The proposed predictive deep Boltzmann machine is evaluated by both hour-ahead and day-ahead prediction experiments based on real wind-speed datasets. The experimental results showed that the proposed deep model outperforms existing statistical approaches and other machine-learning algorithms. Third, deep learning is inspired by the data processing mechanism of human brain; that is, a hierarchical representation of sensory input data. Conjugating with the nonlinear activation functions for neurons in each layer of a deep hierarchical network, deep learning has significant capability to model complex and nonlinear problems. However, the representation capability of existing deep models is still limited due to the communication scheme between neurons. For current deep models, the communications between different neurons are restricted to be constant values, which are different from the nature way of neurons’ interactions in the human brain. As neurons often communicate in an uncertain and varied way, fuzzy sets and fuzzy-logic systems are valid approaches to model the uncertainty and the variation. To improve the capability of deep models and achieve high-level intelligence, fuzzy theory is introduced into deep models in this dissertation. The experimental results also showed the improvement of the proposed fuzzy restricted Boltzmann machine. In summary, this dissertation focuses on above three aspects, that are aimed to develop new deep learning techniques for pattern learning and recognition. All the them are assessed by real-world applications, and the results are published in international journals.