Concept labFoundation80 min estimateOriginal publication

A Neuron Is a Parameterized Decision Surface

Read one neuron as a typed affine measurement, a nonlinear response, and a separate action policy—not as a miniature biological story.

Authorship: InterviewsVector
Published / updated: 2026-08-25 / 2026-08-25
Review status: Artifact tests passing · primary sources recorded

Original InterviewsVector material. Code examples are covered by repository tests and primary references are recorded. No named human reviewer is claimed.

The decision in one pass

A neuron computes a margin z = w·x + b in a declared feature coordinate system, then maps z through an activation. Its weights orient a family of equal-margin surfaces; its bias translates them. The activation changes the response shape, while an action threshold is a downstream policy boundary. Capacity appears when many such units compose: one affine threshold separates one half-space, while nonlinear layers can reuse learned coordinates to form piecewise or curved decisions. Bind feature order, units, semantics, model revision, parameters, activation, threshold, equality convention, and input contents before interpreting any margin or action.

Why this matters

Most single-neuron bugs are ordinary contract bugs disguised as model behavior: swapped feature order, changed units, stale weights, silently different activation semantics, or a product threshold treated as part of training. A geometric reading makes these failures visible and gives an engineer a stable vocabulary for deeper networks.

You will be able to

Derive how weights and bias determine an affine decision surface.
Separate margin, activation, representation, and downstream action semantics.
Explain why nonlinear composition changes capacity while a stack of linear maps remains linear.
Build a content-addressed audit that rejects feature-schema and model-revision drift.
Defend what one neuron can establish and where product policy begins.

Prerequisite contract

Your Vector Loop for this lab

01
Model
Name the feature coordinate system, affine parameters, activation, and action policy.
02
Derive
Derive equal-margin surfaces and distinguish geometry from activation response.
03
Build
Bind typed features, parameters, model revision, and concrete inputs into an executable audit.
04
Stress
Swap order, units, scope, revision, threshold, saturation, and numeric extremes.
05
Operate
Trace margin, activation, and action separately with immutable evidence identities.
06
Defend
State the bounded function represented, the policy applied, and what neither proves.

Model the affine measurement before the activation

Treat x as an ordered, typed representation rather than an anonymous list. A coordinate has a unit and semantics; a weight is meaningful only relative to both. The affine margin is a signed measurement in that coordinate system. Renaming a feature is harmless only when its meaning remains identical, while rescaling dollars to cents without transforming the associated weight changes the function by a factor of one hundred.

z = wᵀx + b

For any constant c, the points satisfying wᵀx + b = c form an equal-margin hyperplane. The normal vector w controls orientation and b controls translation in the chosen coordinates.

Interpretation survives only while feature and model contracts stay fixed.
Quantity	Engineering meaning	Common category error
feature xⱼ	one declared coordinate and unit	assuming position alone carries semantics
weight wⱼ	sensitivity in the chosen scale	reading magnitude across unlike units
bias b	offset independent of observed coordinates	calling it a data feature
margin z	pre-activation signed evidence	calling it a calibrated probability
activation a	nonlinear response g(z)	equating it with a product action

Derive where representation capacity enters

a = g(z); y_action = π(a, τ)

The activation g shapes the unit response. A separate policy π and threshold τ convert a response into an operational action. This artifact binds inclusive equality: sigmoid is positive when z ≥ logit(τ), with 0 < τ < 1, and ReLU is positive when z ≥ τ. Changing τ need not retrain or rotate the affine surface.

Composing affine maps without a nonlinearity collapses into one affine map: W₂(W₁x + b₁) + b₂ = (W₂W₁)x + (W₂b₁ + b₂). Nonlinear activations prevent that collapse. Multiple units can then create reusable regions and features, while later layers recombine those responses. This is a capacity statement, not a promise that training will discover a useful representation.

Activation	Useful property	Diagnostic risk
sigmoid	bounded smooth response	saturation can make local change tiny
ReLU	simple piecewise-linear routing	negative margins produce zero response
hard threshold	explicit half-space decision	no useful ordinary derivative at the boundary

Neuron decision-boundary laboratory

Select a fixed affine-boundary preset and move x1 and x2 to inspect margin, zero-step activation, and geometric classification.

Decision boundary lab

Predict which side of the neuron fires

A two-input neuron first computes an affine score, then applies a step activation that returns 1 at and above zero. Move the input point, inspect the boundary, and commit to a class before revealing the score.

Affine boundary

Boundary ABoth inputs push the affine score in the same direction.Boundary BThe two inputs push the affine score in opposite directions.Boundary CThe vertical input carries three times the horizontal weight.

Input x₁1.00Input x₂-1.00

Zero convention: H(z) = 1 when z ≥ 0, including exactly on the boundary. H(z) = 0 only when z < 0.

Weights: (1, 1)
Bias: −0.500
Boundary: 1x₁ + 1x₂ − 0.5 = 0
Input point: (1.00, -1.00)

zero-score boundaryincreasing-score directioncurrent input

Classification rule: the line is where w₁x₁ + w₂x₂ + b equals zero. The normal arrow points toward larger scores; the numeric score remains hidden until you check.

What does the step activation return for this input?

1 — the neuron activates0 — the neuron stays inactive

Make a prediction, then check it against the current evidence.

Build an auditable decision surface

The reference artifact freezes the exact feature order, units, semantics, weights, bias, activation, threshold, inclusive equality convention, action mapping, scope, model revision, and owner. Input evidence repeats the relevant identities and binds source, schema, observation time, values, and owner into a SHA-256 content identity. The public boundary revalidates concrete frozen records, so a mutable duck type or constructor-bypassed NaN cannot reach arithmetic.

neuron_decision_surface.py

1def format_example() -> str:
2    audit = audit_neuron(ILLUSTRATIVE_CONTRACT, ILLUSTRATIVE_EVIDENCE)
3    return "\n".join((
4        "example=illustrative_only",
5        f"contract_version={ILLUSTRATIVE_CONTRACT.contract_version}",
6        f"evidence_id={audit.evidence_content_id}",
7        f"margin={audit.margin:.3f}",
8        f"activation={audit.activation:.3f}",
9        f"action={audit.action}",
10        f"decision={audit.decision}",
11    ))

Expected output

example=illustrative_only
contract_version=neuron-audit-v1
evidence_id=sha256:48009656b6e97a04e5b03a40185547f126bc7b42d27ff2c4cb77d86270c9f898
margin=1.400
activation=0.802
action=priority-retention-review
decision=PASS

Verify: python3 -m unittest discover courses/ai-engineering/reference-impl/neuron_representation

01Validate identities firstReject wrong scope, model revision, schema, order, duplicates, mutable records, and unsupported numeric values before a dot product is attempted.
02Compute stable layersUse accurate summation and a stable sigmoid branch, reject derived underflow, and decide sigmoid policy in margin/logit space so a rounded displayed activation cannot flip the bound inclusive action.
03Address all evidenceDigest contract semantics and complete input contents, not merely a friendly evidence name.

Stress geometry, numerics, and policy independently

Perturbation	Expected observation	Likely ownership
swap feature order	reject before scoring	feature contract
rescale one unit	margin changes unless weight transforms	representation pipeline
move bias	parallel boundary translation	model revision
change activation	response changes at fixed margin	model architecture
change threshold	action can change at fixed response	decision policy
extreme finite products	reject overflow, underflow, or subnormal results	numeric contract

Operate at three altitudes

Production lens

— Version the feature coordinate system and reject order, unit, or semantic drift before inference.
— Trace pre-activation margin, activation response, downstream threshold, and action as different observables.
— Bind parameters, activation semantics, source snapshot, scope, and input contents into reproducible evidence.
— Treat product threshold changes as owned policy releases even when model bytes remain unchanged.

Staff lens

— Standardize typed feature identities across training, serving, explanation, and incident tooling.
— Separate model release authority from action-policy authority while preserving a joined audit trail.
— Require representation changes to state their invariants, conversions, and backfill plan rather than relying on position-compatible tensors.

Interview defense

What does a single neuron represent, and what changes when you alter its bias or activation?

It computes an affine margin in a declared feature coordinate system. The weight vector is normal to equal-margin surfaces and the bias translates them. The activation maps margin to a response; changing it alters response shape without changing the underlying affine measurement. A downstream threshold maps the response to an action and should be modeled as policy. I would bind feature order, units, parameters, revisions, and evidence, then test schema drift, saturation, numeric extremes, and threshold sensitivity. A single neuron gives one affine measurement; nonlinear compositions create richer capacity.

Expect the interviewer to press on

— Why do stacked linear layers collapse?
— Can a sigmoid output be called a probability?
— How does feature scaling change weight interpretation?
— What would you log during an incident?

Misconceptions to remove

“A sigmoid output is automatically a calibrated probability.”

It is bounded between zero and one, but probabilistic interpretation and calibration require an objective, data, and evaluation claim.

“The largest absolute weight is the most important feature.”

Weights depend on units, correlation, representation, and the local input. Compare controlled effects under the bound schema.

“Changing the action threshold changes the learned representation.”

It can change actions while leaving margin, activation, weights, and representation untouched.

Check your model

1. Why does feature order belong in the scoring contract?

The dot product is positional. Swapping two values applies weights to different meanings even when shapes and numeric types remain valid.

2. What does increasing bias do while weights stay fixed?

It shifts every margin by the same amount and translates each equal-margin surface parallel to itself.

3. Why retain both margin and activation during operation?

They separate the affine evidence from nonlinear response, making saturation and policy-threshold changes diagnosable.

Prove the mechanism

Extend the artifact with an explicitly versioned feature-unit conversion layer. Require a content identity for each conversion and prove with tests that an equivalent scale transformation preserves every margin.

Add a production constraint

Compose a bounded two-layer ReLU network, enumerate its activation regions over a small grid, and produce a content-addressed report that distinguishes discovered piecewise-linear regions from global behavior claims.

Artifact: Neuron representation audit

courses/ai-engineering/reference-impl/neuron_representation/neuron_decision_surface.py

Download reference implementation

Primary references and next links

References

1. A Logical Calculus of the Ideas Immanent in Nervous Activity
McCulloch and Pitts. Primary historical paper defining a formal threshold-unit model.
2. Understanding the difficulty of training deep feedforward neural networks
Glorot and Bengio. Primary paper connecting activation behavior and initialization to signal propagation and trainability.
3. math.fsum
Python documentation. Official documentation for accurate floating-point summation used by the audit.

Continue through the graph

Computation Graphs Make Learning Inspectable →
Turn this scalar computation into explicit forward dependencies.
Vectors as Representations →
Revisit basis, scale, and semantic identity behind the feature vector.
Academy roadmap →
Place the unit inside the full neural-systems capability arc.

Glossary: affine map · margin · hyperplane · activation · saturation · decision threshold · feature coordinate system