Learn a distribution
A generative model describes or approximates a distribution over data. Depending on the model, it may assign likelihoods, generate samples, or both. Conditional generation adds information such as a class label or a text prompt.
Sample, do not copy a promise
A sample is one outcome drawn from a model’s distribution. New-looking output is not proof of novelty, quality or privacy. Models can memorize training examples, and outputs need evaluation. The colored intervals have lengths matching the three probabilities; moving the draw chooses the interval it lands in.
INTERACTIVE EXPLORATION
Sample from a toy model
Move a uniform draw u through a categorical distribution with P(A) = 0.2, P(B) = 0.3 and P(C) = 0.5.
Different paths
Autoregressive models, GANs, variational autoencoders and diffusion models approach generation differently. The interactive example samples from a tiny categorical distribution; it does not train any of them.
Check your understanding
Does one generated sample reveal the whole distribution?
No. It is one outcome; many samples and other evaluation methods are needed.