Backprop Lab 1.2 — watching a neural network learn.
Backprop Lab is a simulator for small neural networks. It classifies points in the plane: every
point has two coordinates x₁ and x₂ and belongs to one of two classes.
The network has to learn to colour the plane so that the colour matches the class.
What runs is real backpropagation with stochastic gradient descent — no video, no animation, no precomputed curve. Every epoch genuinely walks the training data, forms gradients and changes the weights. There is no library behind it; the whole network is some 120 lines of JavaScript in a single HTML file.
The point of it is to see: not just the loss curve falling, but every single neuron cutting out its own share of the plane.
Never worked with neural networks before? Then read the introduction first — it explains neuron, weight, loss and backpropagation from the ground up. This manual takes those terms for granted.
Two versions, one manual. The browser simulator and the Android app both speak German and English, and both are described here. Everything of substance is the same in either: same formulas, same data sets, same colour scale. Section 14 lists what differs on the phone.
x₁x₂ in the feature column —
and try once more.Every control takes effect immediately. Changes to the architecture, the activation or the data set reset the training, because a different network has to be built. Learning rate, batch size and regularisation, by contrast, take hold mid-run.
The controls and the figures sit along the top, and below them three columns: settings on the left, the network in the middle, the evaluation on the right.
| Element | Meaning |
|---|---|
| Train | Starts the training loop, one frame per epoch. The button then reads Pause. |
| Step | Computes exactly one epoch and stops again. Good for following a change move by move. |
| Reset | Fresh random weights, epoch counter to zero, loss curve empty. The data stay. |
| Epoch | Number of complete passes over the training data. |
| Loss test | Mean error on the test points held back — the honest number. |
| Loss train | Mean error on the points that were learned from. |
| Accuracy | The share of test points whose sign the network gets right. |
Top right, beside the buttons, sits a pair of switches, DE and EN. The choice takes effect at once and mid-training: weights, epoch counter and loss curve stay as they are, only the words change. On a first visit the page follows the browser's language; after that the last choice applies. It is kept in the browser's local storage — the only value the page stores at all.
Four tasks, in rising order of difficulty. Clicking a thumbnail draws a fresh sample and resets the network.
| Data set | Rule | What it takes |
|---|---|---|
| Clusters | Two Gaussian clouds around (−2.4 | −2.4) and (2.4 | 2.4). | One straight line is enough — a network with no hidden layer at all manages it. |
| Circle | One class inside, the ring outside. | Needs a hidden layer — or the features x₁² and x₂². |
| Quadrants | Class by the sign of x₁·x₂. |
The classic XOR problem. With the feature x₁x₂ it becomes trivial. |
| Spiral | Two arms wound into each other. | The hard case: several layers, many neurons, patience. |
Noise (0 to 50 %) displaces the points at random before the label is assigned. At high noise some points lie on the wrong side — 100 % accuracy is then out of reach, and forcing it anyway means learning the noise by heart.
Training points (100 to 500) sets how many points are used for learning. The same number again is always drawn on top and held back as the test set; the network never gets to see it. New sample rolls both sets again.
The left column of the network diagram is the input. Not only the raw coordinates are on offer but also combinations worked out in advance. Clicking a tile or its label switches the feature on or off; at least one always stays on.
| Feature | Value | What for |
|---|---|---|
| x₁, x₂ | the coordinates themselves | The basics, on by default. |
| x₁², x₂² | squares | Turn the circle into a straight separation. |
| x₁x₂ | product | Solves the quadrants with a single weight. |
| sin x₁, sin x₂ | sine | Stripe patterns; helps with the spiral, but invites overfitting. |
Features are handwork on the problem. Switch on x₁² and x₂² and the
circle is solved without any hidden layer at all — only then the human did the work, not the
network. That is precisely the difference between feature engineering and deep learning, and
here the two can be laid side by side.
Above the diagram stands the number of hidden layers, zero to five. Above each layer column sits its own counter for the neurons, one to eight. A new layer starts with three.
Without a hidden layer the model stays linear: the inputs go straight to the output neuron. That is enough for the clusters and, with the square features, for the circle. Every further layer folds the space once more — visible in that the tiles in the later layers show more complicated patterns than those in the early ones.
There is always exactly one output neuron and it always uses tanh, so that its
value lies between −1 and +1 and is directly comparable with the label.
| Setting | Range | Effect |
|---|---|---|
| Learning rate | 0.001 – 1 | Step size of the gradient descent. Too small: barely anything happens. Too large: the loss jumps or explodes. 0.03 is a good place to start. |
| Activation | tanh, ReLU, sigmoid, linear | The function inside the hidden neurons. It applies to all of them at once. |
| Regularisation | none, L1, L2 | Penalises large weights. L2 makes them small, L1 pushes many to exactly zero — lines then disappear from the diagram. |
| Strength | 0 – 0.1 | How hard the penalty bites. Without effect as long as no regularisation is chosen. |
| Batch size | 1 – 30 | How many points are averaged before the weights change. Small means jittery and quick, large means calm and sluggish. |
| Function | Behaviour |
|---|---|
| tanh | Soft transitions between −1 and +1, round decision surfaces. The most useful default. |
| ReLU | Cuts off anything negative. Produces angular surfaces assembled from straight lines and learns quickly — but at a high learning rate neurons die off and stay grey for good. |
| Sigmoid | Between 0 and 1, so never negative. Learns noticeably more slowly, because the gradients go small early. |
| linear | No non-linearity at all. However many layers — the result stays a straight dividing line. Well worth trying once. |
Every neuron carries a small picture. It does not show its state for one particular point but its output across the whole plane: for every pixel the question is what the neuron would output if a data point lay there. That is how you see what a neuron responds to.
| Colour | Meaning |
|---|---|
| orange | Strongly negative activation; in the output, class −1. |
| neutral | Close to zero — here the neuron is unsure. In the output surface this is the decision boundary. |
| green | Strongly positive activation; in the output, class +1. |
Features switched off appear dashed and pale and are connected to nothing.
| Line | Meaning |
|---|---|
| green | Positive weight: the neuron takes the signal as it comes. |
| orange | Negative weight: the neuron turns the signal around. |
| thick / thin | The thickness is the magnitude of the weight. Thick lines carry the decision. |
| dashed | Cut by hand — the connection carries nothing and is no longer trained. |
The mouse pointer on a line shows the weight to three decimal places, on a neuron its bias. A click on a line cuts it, a second click puts it back.
Cutting a thick connection while the training runs is the most instructive experiment in the app: the loss jumps up, and you watch in real time as the remaining neurons redistribute the work among themselves.
The large picture on the right is the output of the network over the range −6 to +6 in both directions, with the data points on top. Where the colour under a point matches the point's own, the network has it right.
The shape of a point says what it is for:
| Shape | Meaning |
|---|---|
| filled | Training point — it was learned from. Always visible. |
| ring | Test point — never seen by the network. Appears only with the switch Show test points. |
While only filled points are on show there is nothing to tell apart; as soon as the rings join them, a legend under the surface names both shapes. This is the quickest look at overfitting: if the rings sit as cleanly inside their colour as the filled points do, the network has grasped the rule and not memorised the sample.
Two lines over the epochs: solid green the test loss, dashed grey the training loss. The height of the curve is scaled to the largest value of the first epochs; the number beside it names that scale.
If both lines run downwards together, the network is learning. If the grey one keeps falling while the green one climbs again, it is memorising the training points — and then fewer neurons, more data or regularisation will help.
| Row | Meaning |
|---|---|
| Parameters | Every number that is learned: all the weights plus one bias per neuron. |
| Active connections | Those not cut, out of all connections. |
| Mean |weight| | Mean magnitude. If it grows without stopping, the learning rate is too high or regularisation is missing. |
| Largest weight | The largest magnitude in the network. A single outlier usually means one connection is doing all the work. |
The labels are −1 and +1, and what is measured is half the squared error. An epoch walks the shuffled training points once and changes the weights after each batch.
Forward z = b + Σ wᵢ·aᵢ a = f(z)Here f is the chosen activation, η the learning rate, o
the network output and t the label. The gradients are averaged over the batch, not
summed — which is why the batch size barely changes the learning rate you need.
If regularisation is chosen, an extra pull towards zero is added before every step, with
strength λ:
Under L1, a weight that would shoot past zero is set to exactly zero instead of changing sign. That way the lines in the diagram stay cleanly gone rather than flickering.
Choose Quadrants, activation linear, three layers of eight neurons each. Train. It
stays a straight dividing line and accuracy sticks around 50 %. Back to tanh — the
same architecture solves the task in seconds.
Quadrants, remove every hidden layer, then switch on x₁x₂. The model now has three
weights and is close to perfect after a short while.
Circle, noise at 50 %, training points at 100, three layers of eight neurons, every feature on, batch size 1. Train for a long time and show the test points: the training loss falls towards zero — every training point sits right — the surface grows tongues and islands around single points, and the green test curve climbs again after passing its minimum.
This does not work on every attempt. In measurements with this setting the test loss rose markedly after its minimum in about three out of five samples; with fewer training points it happens more reliably. If nothing happens: New sample and try again.
Then switch on L2 with strength 0.01 and let it keep running. The boundary
smoothes out, the test loss drops to at least half, accuracy climbs — in three runs checked, from
94, 90 and 93 % to 98, 96 and 96 %.
Activation ReLU, learning rate 1, train. Individual tiles turn
completely neutral and stay that way: those neurons output zero for every input, which makes their
gradient zero too, and they never recover.
Features x₁, x₂ plus sin x₁ and sin x₂;
three layers with eight, eight and six neurons; learning rate 0.03, batch size
10. This takes a few thousand epochs — let it run and look in now and again.
| Observation | Cause and remedy |
|---|---|
| The loss barely moves | Learning rate too small, or the task needs more capacity. Raise the learning rate by a factor of ten first, then add neurons. |
| The loss jumps wildly | Learning rate too large. Pick a smaller one or raise the batch size. |
| The surface stays a straight line | The activation is set to linear, or there is no hidden layer. |
| All tiles are neutral | Dead ReLU neurons. Reset and start again with a smaller learning rate. |
| Accuracy will not reach 100 % | At high noise some points lie on the wrong side. That is the limit of the data, not of the network. |
| Test worse than training | Overfitting: fewer neurons, more training points or regularisation. |
The same simulation exists as an Android app — not a web page in a wrapper but a version of its own in Kotlin: network, data sets and training are written a second time, without Android dependencies and covered by unit tests, and the drawing runs through Compose Canvas. It is operated like the browser version, only with a finger: tap a feature to toggle it, tap a connection to cut it.
The app has four destinations, reachable from the bar along the bottom or from the drawer behind the menu symbol at the top left:
| Destination | Contents |
|---|---|
| Lab | The simulation itself — everything the browser version shows. |
| Settings | Language and appearance, see below, plus Share the app and Rate the app. |
| Help | A short form of this manual on the device, boiled down to six sections, and below them the version with its build number, feedback by email, the way to the store rating and the button for the introduction. It sits in the drawer only — with four entries the bottom bar would get cramped. |
| About | A short description, the version number and the links outwards: website, more apps, privacy policy, terms of service. They open the browser with the page in whichever language the app is set to. |
The drawer lists all four destinations by their full names, and below them two handles that otherwise sit far down the lab screen: Reset rolls the weights again, New sample draws new points. Both jump back to the lab afterwards.
Moving between destinations does not stop the training: the network and the compute loop hang on the shell of the app, not on the lab screen. Look into the settings mid-training and come back, and the epochs have gone on counting.
On the very first launch an introduction lies over the app: five pages explaining what a tile shows, what a connection's colour and thickness mean, and why the rings are what count. The pictures in it are not drawings but the same drawing routines as in the lab — the same colour scale, the same way of placing dots and rings.
Skip ends it at once, as does Let’s go on the last page. After that it does not return by itself; at the foot of Help there is Show the introduction again. The app remembers this like language and appearance, so it survives a restart.
The network diagram is a drawing surface and therefore mute on its own. It carries a description instead, naming the current state: how many features are on, what the layers look like, how many connections still stand. The decision surface (with the accuracy) and the loss curve (with the epoch and both losses) do the same, and the figures move along with the training.
The seven feature tiles additionally sit over the drawing as elements of their own: TalkBack reads the name and says on or off, a double tap toggles. Only cutting individual connections stays with the finger on the canvas — as elements that would be two dozen targets nobody could tell apart.
The − and + buttons are named after what they do (“Remove neuron from layer 1”) and can be hit within a 48 dp field, even though they look smaller.
Wherever a label sits above its figure — Epoch and 180, Parameters and 25 — the pair is one stop and is read in one go: “Epoch 180”. Two stops would only mean twice the swiping for the same information. The sliders do the same, naming themselves and their value together.
Which one applies is decided by the window width — not by the class of device. An unfolded foldable gets the same arrangement as a tablet.
| Width | Arrangement |
|---|---|
| under 600 dp phone, portrait |
One scrolling column: buttons, figures, layer controls, network, decision surface, loss curve, status, and the controls for data and training. |
| 600 dp and up phone landscape, tablet |
Two columns: figures, layer controls and network on the left; decision surface, loss curve, status and the controls on the right. Each scrolls on its own. |
In landscape it is not the width that is short but the height. That is why the buttons Train, Step and Reset move up into the otherwise half-empty top bar and stay visible while scrolling; figures and layer controls stand in a single row, and the height of the network diagram takes whatever is left — on a tablet it fills the column, on a phone in landscape it stays at the minimum below which the seven feature slots no longer fit side by side.
Rotating interrupts nothing: the training carries on across the change, weights and loss curve stay as they were. Pushing the app into the background does stop it — no more frames are requested, so nobody is computing either.
Under Settings the display language can be chosen: From system follows the device's language setting, Deutsch and English fix it. The choice takes effect immediately and survives a restart of the app.
The numbers follow it too: in German they read 0,021 and 99,0 %, in
English 0.021 and 99.0% — whatever the device itself is set to.
Below it the same for light and dark: From system, Light, Dark. Switching also recolours the tiles — the zero point of the colour scale sits on the tile colour, and in the dark that is a different one.
The switch happens without restarting the screen — again for a solid reason: a restart would lose the running training along with its weights and loss curve. Instead a specially configured context supplies nothing but different texts, and everything else stays standing.
For Apple devices there is a third version, again not a web page in a wrapper: network, data sets and training are written once more in Swift, line by line after the Android app and with the same tests; SwiftUI does the drawing. One and the same app runs on iPhone and iPad (iOS 17 and up) and as a proper Mac app (macOS 14 and up). Arithmetic, colours, texts and settings are the same as on Android — this section only describes what differs.
The same four destinations as on Android — Lab, Settings, Help and About — but without a drawer:
| Device | Where the destinations are |
|---|---|
| iPhone | In the bar along the bottom, all four — Help included, which on Android sits in the drawer only. Reset and New sample are in the lab behind the … button at the top right. |
| iPad | In the bar along the top, otherwise as on the iPhone. |
| Mac | In the sidebar on the left, with the version and build number below. The handles are in the Lab menu of the menu bar. |
Here too, moving between destinations does not stop the training.
The Lab menu holds everything needed while training. On an iPad with a keyboard the same shortcuts apply.
| Shortcut | Effect |
|---|---|
⌘ ↩ | Train / Pause |
⌘ . | Step — exactly one epoch |
⌘ R | Reset — fresh random weights |
⇧ ⌘ R | New sample |
⌘ T | Show or hide the test points |
On the Mac, features and connections are clicked rather than tapped; the effect is the same.
As on Android the threshold is a width of 600 points: below it one scrolling column (iPhone portrait), above it two columns (iPhone landscape, iPad, Mac window) that scroll on their own. Unlike Android, Train, Step and Reset stay at the top of the left column in the two-column arrangement too. On the Mac the arrangement follows the window width — drag the window narrow and you get the phone view.
On iPhone and iPad the training stops as soon as the app goes to the background. On the Mac it
keeps computing as long as the app runs, even with its window behind others — press
⌘ ↩ to pause it.
The same descriptions as under TalkBack: the network diagram, the decision surface and the loss curve say what is shown, and a label with its figure is one stop. The seven feature tiles can be reached one by one; VoiceOver calls them “selected” while they are on, and a double tap toggles. The − and + buttons can be hit within a 44-point field.
iOS 26.5 · iPhone17,1 or
macOS 26.0 · Mac16,10.The browser version is a single HTML file with no framework and no build step. Network, backpropagation, data sets and every drawing are written by hand; Canvas draws the tiles and surfaces, SVG the connections.
Every tile is a grid of 64 × 64 forward passes over the range −6 to +6. During training the tiles are recomputed every second epoch out of regard for the frame rate; the figures and the curve on every one.
The Android version shares no code with that file, but it does share the arithmetic: same formulas, same data sets, same colour scale. It brings about 1.1 MB with it and likewise computes entirely on the device.
The version for iPhone, iPad and Mac is a translation of the Android app into Swift, with no third-party library: the same arithmetic, the Android app's arithmetic tests plus checks for settings, addresses and number formats, tiles from 48 × 48 forward passes as on Android. On the Mac it runs in Apple's sandbox with no entitlement beyond it.
Everything runs in the browser. There are no accounts, no advertising, no analytics, and no data is transmitted — the only outward request fetches the fonts. The data points are generated afresh by a random generator on every visit. What the apps store on the device is set out in the privacy policy.
| Version | Date | Changes |
|---|---|---|
| 1.2 | 25 Sep 2026 | Browser version: a choice of German or English from a pair of
switches in the top bar, and a footer linking the manual, the privacy policy and the
legal notice. Android app: an introduction on first launch, callable again from the help page; feedback by email and the way to the store rating in the help page, plus a one-off question about rating from the third launch on; Share the app and Rate the app in the settings; descriptions for screen readers on the network diagram, the surface and the curve, toggleable feature tiles, larger touch targets on the stepper buttons and a darker accent in the light theme; links in the About screen to the website, more apps, the privacy policy and the terms; numbers follow the language chosen in the app; the transport buttons survive a larger system font. Also: every page of this manual in English as well, along with a home page, a help page and the legal texts. App for iPhone, iPad and Mac (October 2026): first version, with everything the Android app 1.2 does; on the Mac with a sidebar, a Lab menu and keyboard shortcuts. |
| 1.1 | 11 Sep 2026 | Browser version: a legend for the point shapes, test points as rings
instead of a pale fill. Android app: bottom bar and drawer with four destinations, a help page, a choice of language and appearance, a landscape layout of its own from 600 dp width, data sets as thumbnails. |
| 1.0 | 8 Sep 2026 | First version: four data sets, seven features, up to five hidden layers, four activations, L1/L2, a tile per neuron, cuttable connections, decision surface and loss curve. |