Neural Network Playground

The forward pass

Every point on the plane has two coordinates, X₁ and X₂. Those coordinates (and any extra features you enable, like X₁² or sin(X₂)) are fed into the network's first layer.

Each neuron multiplies its inputs by a set of weights, adds a bias, and passes the result through an activation function — the "Activation" dropdown controls which one. That's the wavy blue-and-gold tile you see inside every neuron: it's literally that neuron's output, plotted for every point in the plane. Early layers tend to learn simple straight-line splits; later layers combine those splits into curves and pockets, which is why deeper networks can fit stranger shapes like the spiral.

Training

Training repeats one loop: run a batch of examples through the network (a forward pass), measure how wrong the output was (the loss, shown in the chart), then nudge every weight a little in the direction that would have made it less wrong (backpropagation + gradient descent). Do this enough times and the tiles converge into the decision boundary on the right. One full pass through the training data is an epoch.

The edges connecting neurons are colored by weight: blue means a positive weight, gold means negative, and thicker lines mean a stronger connection. Watch them during training — a network that's learning has edges visibly growing thicker or changing color as weights move.

Reading the loss chart

The blue line is train loss — error on the data the network is actively learning from. The gold line is test loss — error on data it never trains on, only checks against. If train loss keeps dropping while test loss flattens or rises, the network has started overfitting: memorizing quirks of the training points instead of the underlying pattern. That's usually a sign to reduce network size, add regularization, or add more training data.

The controls

• Learning rate — the size of each weight update. Too high and the loss oscillates or explodes instead of settling; too low and training crawls. If a run looks stuck or erratic, this is usually the first thing to change.
• Activation — the curve applied inside every hidden neuron. Tanh and Sigmoid are smooth S-curves; ReLU is a hinge that's exactly zero for negative inputs; Linear applies no curve, which caps what the whole network can learn no matter how many layers you add (stacked linear layers collapse into one).
• Regularization (L1 / L2) — an extra penalty added to the loss for having large weights, which discourages the network from relying too heavily on any single connection. L1 tends to zero out weights entirely (a form of built-in feature selection); L2 shrinks all weights toward zero evenly.
• Noise — how much random jitter is added to each point before it's labeled. At 0, the true boundary is perfectly clean; higher noise means some points genuinely sit on the wrong side of it, which is what makes real-world data hard.
• Training data % and batch size — how the data is split and grouped. Held-out test data is what makes the test-loss line meaningful; batch size trades update frequency (small batches) against stability (large batches).

A first experiment to try

Switch to the Spiral dataset with only X₁ and X₂ enabled and the default two-layer network — it usually won't fully solve it. Then try adding a third hidden layer, or enabling sin(X₁) and sin(X₂), and watch how much faster the boundary wraps around the arms. It's a quick, visible demonstration of why feature choice and network depth both matter.