Machine learning, one question at a time.

Move a decision boundary and discover what a simple model does.

Explore classification error and the role of surrogate loss.

Understand features, labels, examples, predictions and the difference between training and inference.

See how weights and bias define a decision boundary, and understand why some datasets need nonlinear models.

Explore the linear prediction formula, mean squared error and the limits of a fitted line.

Connect vectors, dot products and matrix multiplication to features, model scores and neural network layers.

Build intuition for a derivative, partial derivatives and gradient directions in machine learning.

Follow a numerical derivative through a small computation graph to understand the chain rule.

Understand probabilities, expected values and the difference between a model score and a calibrated probability.

Experiment with learning rates and watch gradient descent move across a simple loss curve.

See why training performance alone does not establish performance on unseen examples.

Learn the different roles of train, validation and test sets, including leakage and time-based splits.

Explore weighted sums, activation functions and why stacked linear layers alone remain linear.

Follow inputs through weights, biases and activation functions to an output prediction.

Understand computation graphs, local derivatives and the difference between backpropagation and gradient descent.

Explore likelihood, logarithms and negative log loss with a changing probability.

Change logits and see how softmax distributes probability among three classes.

Explore the sigmoid curve and a decision threshold for a binary classifier.

Compare individual-example gradients with a full-batch gradient in a simple two-example problem.

Explore the trade-off in an L2-penalized objective and learn why leakage is a separate data problem.

Try a threshold split and observe predictions, mistakes and weighted Gini impurity.

Explore assignments and within-cluster squared distance for two chosen centers.

Reorder a short sequence and compare the last value with an order-insensitive sum.

Change pixels in a tiny image and apply a fixed edge-detection filter.

Explore a short sequence of rewards and see how discounting changes return.

Explore conditional next-token probabilities and the probability of a short sequence.

Explore a tiny token lookup and how a projection scores the resulting embedding.

Slide a small filter over a one-dimensional signal and inspect every output.

Adjust a residual branch and observe the combined output and gradient.

Hide part of a short sequence and inspect a target created from the data itself.

Choose an action in a tiny deterministic environment and trace its next state and reward.

Move a random-number draw through a discrete cumulative distribution.

Adjust two candidate scores and observe a simple pairwise preference probability.

When a word has more than one meaning, its neighbors can help. How can a model combine information from those neighbors?

A sequence arrives one piece at a time. How can a model carry information from earlier steps?

A small pattern can appear in many places. Can we look for it using the same detector at each location?

A decision may pay off later. How can an agent compare rewards arriving at different times?

A classifier predicts a label. What if we want a model to generate an example instead?

Suppose people prefer one answer over another. Can a model learn a score that reflects those comparisons?

Two classes can be separated in many ways. Which boundary leaves the widest gap between them?

What if a dataset has many coordinates, but most of its variation follows one direction?

Translation takes words in one language and produces words in another. How can the input and output have different lengths?

Attention combines information across tokens. How do we turn that operation into a model for sequences?

An agent can choose actions, but how can it judge what those choices may lead to?

Given an image, can a model choose among a fixed set of categories?

A language model sees a prefix and predicts what comes next. How can that small task create useful representations?

As networks get deeper, training can become harder. Can a shortcut give information another path?

Training from scratch can need lots of data. What if a model has already learned useful features?

Image classification can say “cat”. What if we also need to know where the cat is?

A box gives a rough location. What if we need a label for each pixel?

Can a few hidden numbers capture useful variation in many observed values?

A value summarizes future rewards. Can we express it using one step and the value after that step?

A policy assigns action probabilities. How can feedback change those probabilities?

How can a model learn a useful representation by comparing examples?

A transformer reads tokens. What if those tokens describe pieces of an image?

How could a model connect an image of a dog with the words “a dog”?

Can a model learn a useful hidden representation by trying to reconstruct its input?

What if an encoder describes a distribution over hidden values rather than one fixed point?

What happens if one network makes examples and another tries to distinguish them from real ones?

If we learn to reverse a gradual noising process, can we generate new data?

Can an agent learn about future reward before a whole episode ends?

What if we estimate the future value of each action in each state?

A table becomes impractical in a large state space. Can a neural network estimate action values?

An action earned a reward. How can we judge whether it was better than expected in that state?

A policy chooses actions. Could another component help judge those choices?

A policy update can be too large. Can we discourage excessively large probability changes?

Before acting, could an agent use a model to predict possible outcomes?

After pretraining, how can examples demonstrate the kind of response we want?

If people prefer some answers to others, how can that feedback affect model behavior?

Could we use a preferred and a rejected answer without training a separate reward model?

A model looks better on one example. How do we tell whether the change generalizes?