Deep Learning
Deep learning is a subset of machine learning that uses artificial neural networks with many layers — 'deep' refers to the number of processing layers — to learn complex patterns directly from raw data without requiring human-designed features. Enabled by GPU computing and large datasets, deep learning has achieved human-level or superhuman performance in image recognition, natural language processing, and game-playing since 2012. It is the core technology powering modern AI: ChatGPT, image generators, self-driving systems, and protein-folding models are all built on deep learning architectures.
A deep learning model consists of layers of artificial neurons organized in a directed graph. The input layer receives raw data (pixels, text tokens, audio samples). Each subsequent hidden layer applies a mathematical transformation — a weighted combination of its inputs followed by a nonlinear activation function — passing its output to the next layer. The output layer produces the final prediction. 'Deep' refers to having many hidden layers: a simple neural network might have 2-3 layers; modern large language models have dozens to hundreds.
Training a deep learning model requires three ingredients: a large dataset (millions to billions of examples), a loss function (a measure of how wrong the model's predictions are), and an optimization algorithm (typically stochastic gradient descent with backpropagation). The training process adjusts the millions or billions of numerical weights in the network to minimize the loss function. After training, the weights encode whatever statistical regularities the network found most useful for making accurate predictions.
The deep learning revolution was triggered by three coincident developments: (1) the availability of large labeled datasets (ImageNet, with 14 million labeled images); (2) the adaptation of GPU processors — originally designed for video games — to parallelize the matrix multiplications at the core of neural network computation; and (3) algorithmic improvements, particularly the development of dropout regularization and ReLU activation functions, that made very deep networks trainable. AlexNet's 2012 victory in the ImageNet competition — achieving 26% lower error than the nearest competitor — marks the beginning of the deep learning era.
Professor Jiang uses deep learning as a case study in the black box problem. Deep learning systems can exceed human performance on narrowly defined tasks, but their success comes at the cost of interpretability: the hundreds of millions of weights encode statistical patterns that cannot be read as rules or principles. This is why, Professor Jiang argues, the claim that deep learning will produce artificial general intelligence is structurally incoherent: AGI requires general reasoning, but deep learning produces statistical pattern-matching that breaks down at the edges of its training distribution.
Related articles
Related terms
Frequently asked questions
What is deep learning? ＋
Deep learning is a type of machine learning that uses neural networks with many layers to learn patterns directly from raw data. 'Deep' refers to the number of layers: modern systems can have dozens to hundreds. Each layer learns increasingly abstract representations of the input data, from simple edges (in image recognition) to complex semantic concepts. Deep learning has achieved human-level or superhuman performance in image recognition, language translation, game-playing, and protein-folding prediction. It is the core technology behind ChatGPT, DALL-E, self-driving systems, and virtually all modern AI applications.
What is the difference between deep learning and AI? ＋
AI is a broad field encompassing any system that performs tasks typically requiring human intelligence. Machine learning is a subset of AI that learns from data rather than being explicitly programmed. Deep learning is a subset of machine learning that uses multi-layer neural networks. Modern AI in practice is almost synonymous with deep learning: the systems that have achieved the most impressive results — language models, image generators, game-playing systems — are all deep learning models. However, deep learning has a fundamental limitation: it learns statistical patterns, not principles, making it brittle outside its training distribution and unsuitable for genuine general reasoning.