II · Specification

Ledger notation

A presentation and synthesis of existing shape-aware, named-tensor ideas, not a claim to invent named axes or contraction. We put a tensor’s shape directly beneath its symbol, the way a quantity carries its units.

Our main antecedent is Chiang, Rush & Barak, Named Tensor Notation (first posted 2021; version 3, 2023, checked here). Named axes, lifted operations, named contractions, renaming and the recurrent input/output distinction all come from that work. Our choices are a shape-under-symbol display, explicit virtual axes, separate dot/circle glyphs, and the local elision policy below.

What the alternatives trade off

Bare letters

h_{t} = o_{t} ⊙ tanh c_{t}

Compact and familiar. A paper can define the missing information in prose, but an isolated equation does not tell you whether h is a vector or a batched matrix. We want the information on the line, not only in surrounding text.

Tensor type declarations

h_{t} \in R^{B \times D}, W \in R^{E \times K}

These can be fully precise. Their drawback for this presentation is distance: a declaration may be far from its use. We repeat types locally rather than claiming declarations are mathematically inadequate.

Indices, Einstein sums and einsum

z_{bk} = e \sum x_{b e} W_{e k} + d \sum h_{b d} U_{d k} + b_{k}

Exact and executable as einsum. Ranges can be declared to make sizes explicit; mnemonic indices can carry semantics. Dense component notation is nevertheless cumbersome across nonlinearities. Einstein conventions differ: a repeated batch index must not accidentally become a summed index. Our dot writes the summed axis explicitly.

Named tensor notation

Chiang–Rush–Barak already define name/size axes (§2), including $ax [n]$ , optional size omission when unique (§2.1), broadcasting/lifting (§§3.1–3.2), reductions (§3.3), contraction (§3.4), renaming (§3.6), and an RNN with distinct hidden and hidden′ axes renamed for recurrence (§4.2). They also give attention (§4.3, Eqs. 9–10) and distinguish derivative input/output axes (§5). These are strengths we adopt, not omissions we repair.

Our stricter explicit-broadcast syntax is a teaching choice; their implicit lifting has a formal definition. Our dot-versus-circle glyphs distinguish contraction from element-wise multiplication visually, while their contraction is an axis-labelled ⊙. Short capitals are optional; descriptive word names work equally well. Shape-under-symbol ink and the E1–E5 policy are our local presentation convention, not a new tensor calculus.

Code type annotations and underset shapes

Shape-typed code can catch mistakes automatically. Underset shape labels are already a familiar explanatory device. Ledger combines those ideas with named-axis algebra and a reproducible local notation policy; it does not replace a type checker.

The ledger

Axis identity	Meaning	Size
$B$	independent sequences in a batch	16
$T$	time steps	128
$E$	input embedding coordinates	512
$D$	recurrent state coordinates (source of the recurrent projection)	768
$R$	projection output coordinates, distinct from D; canonical coordinate-wise bijection R → D	768
$G$	gate labels, in order i, f, g, o	4
$K = G \cdot R$	fused gate-output axis; an atomic identity until unfused	3072 = 4·768
$J = E + D$	input and recurrent state concatenated, in that order	1280

Sizes are not axis identities. R and D both have 768 entries but cannot be silently matched. K has 3072 entries (often called “4D” in code); it does not share a free D or R axis with another tensor. A size-only numerical view is a check against this ledger, never a replacement for its identities.

Rules

L1 · Identity is separate from size

Every axis has a name, cardinality and coordinate set. Equal cardinalities do not imply equal identity. A tensor may contain an axis name at most once. Thus a dense recurrent map is $D \times R U^{i}$ , never D×D. D is its source and R its destination.

L2 · Attach the shape

B \times D h_{t}, D \times K U, B \times T \times D H

The written axis order records storage order, but axes match by name. An identity-preserving reordering changes layout, not values. A scalar has empty shape. A numerical-only view, such as 16×768, is allowed only with the ledger in scope: it cannot resolve same-sized identities by itself.

L3 · Contract exactly the named axes

B \times K a = B \times E x E \cdot E \times K W, E \times K δ W = B \times T \times E X B, T \cdot B \times T \times K δ Z

Each contracted axis must occur once in each operand and have matching identity and cardinality. Multiply and sum over those indices. Other shared axes are matched pointwise and retained once; other axes are retained independently. Output storage order is retained shared axes, then remaining left-only axes, then right-only axes. The sum axis is bold gold; shape ink is oxblood. Equality is of named tensors, not necessarily their memory layouts.

L4 · Element-wise operations

B \times D c_{t} = B \times D f_{t} ⊙ B \times D c_{t - 1} + B \times D i_{t} ⊙ B \times D g_{t}

⊙, + and − require identical effective axis sets, except for scalars. Element-wise functions preserve shape. A circle does not sum. This is a statement about dependency structure, not a guarantee against vanishing gradients.

L5 · Explicit broadcasting

B \times K a + (B) \times K b

A parenthesised axis is virtual: broadcast a stored K-vector over B. Parameter counts omit virtual axes. Scalars may broadcast freely. Other missing axes are errors unless explicitly marked virtual. A gradient of the stored b is K, obtained by summing B, not a gradient of duplicated bias parameters.

L6 · Indexing and reductions

B \times E x_{t} = (B \times T \times E X)_{t}, B \times D = T \sum B \times T \times D H

An index selects a coordinate and removes that axis. Every reduction names the axes it removes. Softmax names its normalised axis and keeps it.

L7 · Renaming is explicit

ρ_{R \to D} : B \times R v \mapsto B \times D v_{[R \to D]}

ρ changes an axis identity through a declared coordinate-wise bijection, with no arithmetic on values. Its inverse is ρ from D to R; backpropagation uses that inverse. Same-size names may never be silently conflated. For an unfused gate:

B \times D i_{t} = σ (ρ_{R \to D} (B \times E x_{t} E \cdot E \times R W^{i} + B \times D h_{t - 1} D \cdot D \times R U^{i} + (B) \times R b^{i}))

L8 · Fusing, splitting and concatenating

B \times K z_{t} split_{K \to (G, R)} B \times G \times R z_{t}

K = G·R is a fused atomic axis with gate-major bijection k = g·768+r (zero-based). Select gate g to get B×R, then rename R→D. We abbreviate this explicitly defined composite operation as:

(B \times D q_{t}^{i}, B \times D q_{t}^{f}, B \times D q_{t}^{g}, B \times D q_{t}^{o}) = unpack_{K \to G, R \to D} (z_{t})

The inverse, pack, renames D→R and fuses G·R→K in the same order. Concatenation gives J=E+D via its declared disjoint-union ordering, input first, state second:

B \times J v_{t} = cat_{E, D \to J} (x_{t}, h_{t - 1}), J \times K V = [W U]

L9 · Recurrence and gradients

B \times T \times D H = stack_{T} (h_{1}, \dots, h_{T}), D \times K δ U = B \times T \times D H_{-} B, T \cdot B \times T \times K δ Z

A scalar-loss adjoint δX has the stored shape of X. A full tensor-to-tensor Jacobian needs distinct input/output copies of every shared axis (Chiang–Rush–Barak §5); the simple gradient convention is not a definition of all Jacobians.

Short form: E1–E5

E1. Declare each symbol’s shape in the local block, or inherit it from the ledger and an explicit family declaration.

E2. Repeated occurrences may omit unchanged shapes. A time slice inherits the declared parent’s shape minus T.

E3. Every newly defined result keeps its shape; a tuple may use a previously declared shape family.

E4. A contraction’s axis label may be omitted only when both operand types share exactly one real axis. Two shared axes (such as B and F in attention) require a label.

E5. Renames, pack/unpack, concatenation, stacks and reductions retain explicit axis mappings, even in short form. A declared virtual bias axis may subsequently be omitted but never inferred merely from its size.

By hand: write a small shape below the symbol, a dot with its summed axis below, and a circle-dot for element-wise multiplication. R→D is a short arrow; unpack’s subscript records the rename. The cost is a little more ink in return for an explicit type check.

For the exact local rendering macros see src/lib/tex.js. Two key commands are \sh{B\times D}{h_t} and \dt{E}. This notation is a teaching convention, not a standard.