Open almost any deep-learning paper and you meet a lower-case h. It looks like a number. It is a vector of 768 numbers, and when the network is trained, which is the only time anyone runs the equations, it is a matrix of 16 × 768, one row per example in the batch. None of that is on the page. The reader rebuilds it from the prose, from the code, or from a footnote, and gets it wrong often enough to lose an afternoon.
This site presents ledger notation, our synthesis of existing shape-aware and named-tensor ideas. Every tensor carries its shape written beneath it, the way a physical quantity carries its units. Each product says which axis it sums over. Broadcasting, concatenation and splitting are written down instead of assumed. Once a shape has been declared, a precise rule says when you may stop writing it, so the short form stays as light as the old notation.
Read it aloud: z is batch by fused gate-output axis K (3072); x dotted with W over the input axis, plus h dotted with U over the hidden axis, plus a bias that is stored once and repeated down the batch. The ledger also fixes the gate order and K’s split into G×R, with R explicitly renamed to recurrent coordinates D.
Contents
- iWhat is h?
An LSTM as papers print it, then the same equations with every hidden shape exposed.
- iiLedger notation
What the existing notations get wrong, and the eleven rules that replace them.
- iiiThe forward pass, with its shapes
One time step, then the whole sequence, at B = 16, E = 512, D = 768, T = 128.
- ivBackpropagation through time
Every gradient wears its variable’s shape. Where batch and time are summed, and when the direct cell path helps or vanishes.
- vShape explorer
Move B, T, E and D. The equations, the parameter count and the blocks change with you.
- viTranslating papers
Hochreiter & Schmidhuber 1997, Graves 2013, the GRU and an attention head, old against new.
- viiPrimary sources & corrections
Precise equation references, adaptations and the source-check record.
The running example throughout: batch , sequence length , input width , hidden width , four gates fused into . Forward/backward shapes and counts were checked against a NumPy implementation, including the backward pass against finite differences. Primary sources and adaptation notes distinguish historical equations from our modern teaching model.