5.2 Score matching and Langevin dynamics
The gradient of the log-density points toward more probable data, never needs the normalising constant, and with injected noise turns into a sampler: Langevin dynamics, the engine inside score-based diffusion.
01Generative modelling and the normaliser problem
A generative model has two jobs: assign a probability to any data point , and produce new samples from it. Flexible models usually give the first job only up to a constant. A network outputs a score of plausibility (an energy-based model), and the probability is that number divided by its total over every possible input:
Z integrates over all of ℝ^d. A grid with only 10 points per axis needs 10^d evaluations, so for images Z cannot be computed, and neither can likelihoods or exact samples that depend on it.
The viewport draws a two-dimensional stand-in: a mixture of three Gaussians, A with weight 0.2 at and spread , B with weight 0.3 at and C with weight 0.5 at , both with spread . The emerald surface is the density; the emerald rings on the floor mark one standard deviation of each component. Here is known, but this lesson never uses it. The 500 white dots are sampler chains that start spread evenly over the box and only ever look at the amber arrows on the floor.
Restart the sampler and hold it at step 0, before any chain has moved.
Deep Learning, Goodfellow, Bengio & Courville, Ch. 16.2.4 and 18.1
Maths, CS & AI Compendium, Henry Ndubuaku
02The score function, and why it ignores Z
The score of a density is the gradient of its logarithm with respect to the data (not the parameters, as in statistics' Fisher score):
- 1
Taking logs turns the division by Z into a subtraction.
At , half a unit to the right of C's mean, C's responsibility is essentially 1, so the score is : back toward C. Far from every mode the arrows still point inward, but the drawn length saturates (it grows like ) so the field stays readable.
Deep Learning, Goodfellow, Bengio & Courville, Ch. 18.4
Score-Based Generative Modeling through SDEs, Song et al., 2021, Sec. 2.1
03Langevin dynamics: climb the log-density, plus noise
Following the arrows alone is gradient ascent on : every chain climbs to the nearest mode and stops. A sampler must also spend time on the slopes, in proportion to how probable they are. Langevin dynamics adds exactly enough Gaussian noise to do that:
The two terms balance. The drift pulls a chain uphill by an amount proportional to ; the noise kicks it by an amount proportional to , which is larger for small steps. Near a peak the drift is weak and the noise wins, so the chain wanders; far out on a flank the drift is strong and wins. The result is a chain that lingers where is high and visits where is low, at the right rates.
- 1
As ε → 0 the update becomes this stochastic differential equation: ε plays dt, and √ε z is a Brownian increment dW.
Watch the viewport: within about 100 steps each chain has joined a cloud, and by step 300 the clouds have the right shape. The per-axis spread inside B and C is 0.41 and 0.43, against a target of 0.42 (the 0.4 of each component, inflated slightly by the finite step; see the warning below). But read the occupancy in the HUD: 37 · 30 · 33 % against weights 20 · 30 · 50 %. A holds almost twice its share and C two thirds of its share.
Remove the term and restart: pure gradient ascent on .
Compare the two runs. With noise the counts moved only from 46 · 30 · 24 to 37 · 30 · 33: each chain mostly stays in the basin it started in, because crossing from A to C means passing through a valley where is about 30 times lower than at A's peak and 130 times lower than at C's, and kicks of typical size against a steady drift back almost never carry a chain through it. Langevin dynamics does converge to , but the time to move mass between well-separated modes grows exponentially with the barrier. The same 500 chains run for 1,200 steps instead of 300 reach 27 · 27 · 46 %: closer, still not there. Step 5 fixes this.
Raise the step size to , five times the default.
Score-Based Generative Modeling through SDEs, Song et al., 2021, Sec. 2.1
Deep Learning, Goodfellow, Bengio & Courville, Ch. 17.3
The rest of this lesson is in Pro
- 04Score matching: learning the arrows from data
- 05Noise-conditional scores and annealed Langevin
- 06Bridge to diffusion: the SDE view and ε-prediction
- Python challenge: Score of a mixture, a Langevin sampler and the DSM target
- Knowledge check, 4 questions
Pro opens all 45 lessons, every graded Python challenge and the full 3D labs.
The Python challenge and knowledge check for this lesson are part of Pro.
Questions learners ask
- What is the score function in diffusion models?
- The score is the gradient of the log-density with respect to the data, a vector field that points toward more probable data. It does not depend on the normalising constant, so a network can learn it without ever computing a likelihood. Diffusion models learn the score of the data blurred by every noise level, and generate by following it from pure noise back to data.
- What is the difference between score matching and denoising score matching?
- Explicit score matching fits the score directly but needs the trace of the network's Jacobian, which costs one backward pass per input dimension. Denoising score matching adds Gaussian noise to each training point and regresses the network onto the known score of that noise, minus the noise divided by sigma. It costs one forward and one backward pass and learns the score of the noise-blurred data.
- How are score-based models and DDPM related?
- They are two discretisations of the same idea. Score-based models with annealed Langevin dynamics use a ladder of noise levels and a score network; DDPM uses a variance-preserving noise schedule and a network that predicts the added noise. The noise prediction equals minus the noise standard deviation times the score, and Song et al. showed both are reverse-time solvers of a stochastic differential equation.
Next in Generative AI and Diffusion
5.3DDPM: the forward and reverse processesA fixed noising Markov chain, a learned denoiser, and the simple ε-prediction loss that ties them together.Sources
- Score-Based Generative Modeling through SDEs, Song et al., 2021
- Maths, CS & AI Compendium, Henry Ndubuaku
- Denoising Diffusion Probabilistic Models, Ho, Jain & Abbeel, 2020
- Deep Learning, Goodfellow, Bengio & Courville