Normalization and Residual Paths

Batch normalization and layer normalization do not normalize the same population. A batch statistic couples examples and changes between training and inference unless running estimates are used; a layer statistic is computed within an example over declared features. Pre-normalization and post-normalization also place the identity path on different sides of the transform. Record axes, epsilon, affine parameters, training mode, and placement instead of logging only norm=layer.

1. 01Name the tensor boundaryChoose pre-activation, post-activation, residual input, residual branch, or block output; these are not interchangeable samples.
2. 02Freeze the windowBind checkpoint, data version, batch, training steps, precision, and sampling rule so adjacent layers describe one execution context.
3. 03Preserve orderRequire every declared layer exactly once. A sorted dashboard can silently compare the wrong neighbors.
4. 04Pair forward and backward evidenceActivation stability with vanished gradients and gradient stability with saturated representations imply different next tests.

1    priority = {"UNSUPPORTED": 0, "EXPLODING": 1, "VANISHING": 2}
2    status, first = "STABLE", None
3    for observation, flags in zip(evidence.observations, flags_by_layer):
4        if flags:
5            status = min(flags, key=priority.__getitem__)
6            first = observation.layer_id
7            break
8    return SignalPathReport(
9        contract_content_id=contract.content_id,
10        evidence_id=evidence.evidence_id,
11        evidence_content_id=evidence.evidence_content_id,
12        status=status,
13        first_flagged_layer=first,
14        activation_ratios=tuple(activation_ratios),
15        gradient_ratios=tuple(gradient_ratios),
16        causal_claim="TELEMETRY_ONLY_NO_CAUSAL_ATTRIBUTION",
17    )

example=illustrative_only
contract_version=signal-path-v1
evidence=signal-window-001
evidence_content_id=signal-evidence@sha256:2a85c38d91c96fbeb4b703166b859bcd01c026dac07eb5711603a61a275d8bfc
layers=3
activation_ratios=0.900,0.900
gradient_ratios=1.100,1.100
status=STABLE
claim=TELEMETRY_ONLY_NO_CAUSAL_ATTRIBUTION

The excerpt is literal artifact source. Before classification, the public boundary revalidates concrete frozen records; recomputes the contract and complete evidence hashes; enforces architecture, checkpoint, data, batch, precision, normalization, residual, and window scope; requires exact layer order and coverage; rejects duplicate observation and layer identities; gates sample support; and rejects boolean, non-finite, negative, sub-resolution, or excessive statistics. The earliest flagged layer determines the reported boundary. When one layer has several symptoms, unsupported takes precedence over exploding, which takes precedence over vanishing.

1. 01Remove a middle layerReject incomplete coverage instead of joining the remaining points into a fictitious adjacent ratio.
2. 02Reuse one observation IDProve support cannot be manufactured by copying a record under the same or a second layer identity.
3. 03Zero the baseline varianceReturn unsupported for the affected ratio; do not divide by an epsilon chosen after seeing the run.
4. 04Change only precision or residual placementInvalidate the evidence-contract match because the execution path has changed even if layer names are identical.
5. 05Attack numeric resolutionReject booleans, NaN, infinity, subnormal nonzero summaries, unsafe thresholds, and magnitudes that could overflow ratio arithmetic.

Store the probe definition beside the checkpoint and training configuration. Sample enough to preserve distributions without making every step a full-tensor synchronization. Alert on sustained, supported movement rather than one noisy mini-batch, and retain the first failing batch plus random, optimizer, scaler, and data-loader state for replay.

• State exactly which tensor boundary, axes, population-variance convention, window, and support produced each moment.
• Call a ratio stable only inside the predeclared interval and report zero-denominator paths as unsupported.
• Describe vanishing or exploding telemetry as a symptom location, not a diagnosis of normalization, initialization, or residual design.
• Use controlled replay or ablation to discriminate mechanisms, then validate task behavior and resource cost independently.