A set loses order
The sequences [1, 2, 3] and [3, 2, 1] have the same sum, but different last elements. Any representation that keeps only the sum cannot distinguish them.
Inputs, outputs and context
Sequence tasks include predicting a next value, classifying an entire sequence and mapping one sequence to another. A model needs a way to represent which information came before and, for some tasks, after. Switch the order and compare the last position with the sum. A sum alone loses order even though the sequence still contains the same values.
INTERACTIVE EXPLORATION
Reorder the same values
Switch between two sequences with identical values. Compare a sum-only representation with the last element.
Different architectures, different mechanisms
Recurrent models carry a state across steps. Transformers use attention and position information. Neither architecture guarantees that the model understands a sequence; performance depends on data, objective and evaluation.
Check your understanding
Can a model that sees only the sum distinguish [1, 2, 3] from [3, 2, 1]?
No. Both sums are 6, even though their order and last elements differ.