Encode, then decode
An encoder maps input x to a latent representation z. A decoder maps z to a reconstruction x̂. A reconstruction loss measures their difference.
A bottleneck is a constraint
A smaller latent representation can limit what is retained, but a useful representation is not guaranteed. Without suitable constraints, a model can learn an unhelpful copying strategy. Follow the input through the mean bottleneck to its reconstruction. This fixed rule is not trained.
INTERACTIVE EXPLORATION
Reconstruct from a mean
Change x₂. Reconstruct both input coordinates as their mean.
Experiment with an approximation
Our toy decoder reconstructs [x₁,x₂] using their mean for both coordinates. It illustrates compression error; it is not a trained autoencoder.
Check your understanding
If the input is [2, 4], what is its mean-based reconstruction?
[3, 3].