Concept labFoundation95 min estimateOriginal publication

Probability for Decisions, Not Decoration

A probability becomes operational only when its conditioning context is valid, its frequency meaning is calibrated, and a versioned policy converts it into an action.

Authorship: InterviewsVector
Published / updated: 2026-08-14 / 2026-08-14
Review status: Artifact tests passing · primary sources recorded

Original InterviewsVector material. Code examples are covered by repository tests and primary references are recorded. No named human reviewer is claimed.

The decision in one pass

A model probability should estimate a conditional event frequency for a named population and time horizon; it is not an action and not a generic confidence badge. Check calibration on data independent of model fitting, update beliefs only with justified evidence, then choose the action with minimum expected cost under an explicit, versioned cost table. Thresholds therefore belong to decision policy: when prevalence, calibration, costs, constraints, or review capacity changes, the correct action can change even if the model score does not.

Why this matters

Teams often optimize classification accuracy and then attach 0.5 as if it were a universal business threshold. This hides asymmetric harm, stale base rates, uncalibrated scores, and human capacity constraints. The system can improve its model metric while making more expensive decisions.

You will be able to

State a probability as a conditional claim over a named event, population, context, and horizon.
Use prior odds and a likelihood ratio to derive a posterior without ignoring base rates.
Distinguish discrimination, calibration, and decision utility and select evidence for each.
Compute expected action costs and derive where a binary policy changes decisions.
Build a versioned, fail-closed decision policy with calibration evidence and immutable traces.

Prerequisite contract

— Fractions, averages, and basic algebra
Define the Measurement Before the Model →

Your Vector Loop for this lab

01
Model
Define the event, conditioning context, horizon, candidate actions, and consequences.
02
Derive
Update odds, inspect calibration, and derive expected cost for every action.
03
Build
Encode versions, cost rows, tie behavior, probability validation, and calibration evidence.
04
Stress
Shift prevalence and costs, inject invalid scores, create ties, and slice calibration failures.
05
Operate
Monitor probability quality, policy decisions, realized cost, capacity, and version lineage.
06
Defend
Separate what the model estimates from why the product takes a particular action.

Model a probability as a conditional claim

p = P(Y = 1 | X = x, population, time, measurement process)

The number depends on what event Y means, which evidence X was available, who is in scope, the prediction horizon, and how outcomes are observed. Remove that context and p loses its operational meaning.

Field	Concrete example	Failure if omitted
event	refund request later confirmed abusive	teams optimize different labels
horizon	within 30 days of request	unfinished cases become false negatives
population	authenticated consumer refunds in one region	base-rate shift is hidden
evidence time	features available before triage	future leakage inflates confidence
measurement	appeal-adjusted adjudication	selective labels masquerade as truth

A probability is not uncertainty about anything in general. It is a claim about one event under declared conditions. A score can preserve rank yet fail to be a probability; a calibrated probability can still be useless if the event definition or observed labels do not match the product decision.

Derive the update without losing the base rate

posterior odds = prior odds × likelihood ratio

Prior odds encode prevalence before new evidence. The likelihood ratio says how much more likely that evidence is under the event than under no event. Multiplication makes the contribution of each explicit.

P(Y|E) = P(E|Y)P(Y) / P(E)

Bayes' rule reverses the conditioning direction. P(Y|E) cannot be inferred from P(E|Y) alone; the event base rate and evidence frequency matter.

If an event has prior probability 0.2, its prior odds are 0.2/0.8 = 0.25. Evidence with likelihood ratio 4 produces posterior odds 1 and posterior probability 0.5. Saying the evidence is four times more likely under the event does not mean the event probability is 0.8.

Test whether the number behaves like a probability

Calibration is a frequency claim: among comparable cases assigned probabilities near p, the event should occur at roughly rate p. A reliability diagram groups predictions, plots mean predicted probability against observed event rate, and shows how much data supports each region. Inspect it by time, population, language, model version, and decision pathway because aggregate agreement can hide offsetting slice errors.

Brier = (1/N) Σᵢ (pᵢ − yᵢ)²

The Brier score is a proper score for binary probabilistic predictions, but it combines reliability, resolution, and outcome uncertainty. Use it with reliability evidence rather than treating one scalar as a complete calibration diagnosis.

Property	Question	Evidence
discrimination	do higher scores rank events above non-events?	ROC/PR behavior and slice ranking
calibration	does p correspond to observed frequency?	reliability bins, proper scores, uncertainty
decision utility	do actions improve the intended outcome at acceptable cost?	policy simulation and online outcomes

Derive actions from expected cost

EC(a | x) = p(x) C(a, event) + (1 − p(x)) C(a, no event)

For each allowed action a, weight its consequence in each possible outcome by the conditional probability. Choose the minimum expected cost only after validating that the costs and probability apply to this context.

Suppose auto-clear costs 12 when abuse is present and 0 otherwise, while manual review costs 2 regardless of outcome. Auto-clear has expected cost 12p; review has cost 2, so review becomes cheaper above p = 1/6. The threshold came from consequences. Changing review cost, missed-event harm, or available actions changes the threshold without retraining the model.

Policy input	Owner	Why it changes
calibrated event probability	model and calibration owners	population, data, or model shifts
action set and authority	product and operations	new workflow or regulatory constraint
cost table	domain, finance, safety, policy	measured impact or value judgment changes
capacity constraint	operations	review queue and service level vary
tie/abstain behavior	policy owner	ambiguity requires an explicit safe path

Build a versioned cost-sensitive policy

The reference artifact refuses naked probabilities. Each immutable estimate carries its own ID, event, population, horizon, model, calibrator, evaluation data, evaluation ID, and evaluation window. Calibration observations share one immutable cohort contract; mixed scopes and duplicate case IDs are rejected, bin allocation is bounded before allocation, and the resulting report must match the estimate and policy exactly. The policy version also sets the calibration bin count, minimum calibration count, maximum Brier score, maximum calibration gap, accountable owner, cost-table revision, immutable action costs, and a bounded tie tolerance that callers cannot override. A decision is emitted only after every provenance and calibration-quality gate passes, and its trace preserves the observed values and thresholds.

cost_sensitive_policy.py

1if __name__ == "__main__":
2    report = calibration_report(EXAMPLE_OBSERVATIONS, bins=2)
3    trace = decide(EXAMPLE_POLICY, EXAMPLE_EVIDENCE, report)
4    print(f"policy={trace.policy_id}")
5    print(f"event_probability={trace.event_probability:.3f}")
6    print(
7        "expected_costs="
8        + ",".join(f"{name}:{cost:.3f}" for name, cost in trace.expected_costs)
9    )
10    print(f"decision={trace.action}")
11    print(f"brier={report.brier_score:.3f}")
12    print(f"max_calibration_gap={report.max_calibration_gap:.3f}")
13    print(
14        f"scope=event:{trace.event_definition} population:{trace.population} "
15        f"horizon:{trace.horizon}"
16    )
17    print(
18        f"versions=model:{trace.model_version} "
19        f"calibration:{trace.calibration_version} "
20        f"data:{trace.data_version} evaluation:{trace.evaluation_id} "
21        f"costs:{trace.cost_table_version}"
22    )
23    print(f"evaluation_window={trace.evaluation_window}")
24    print(
25        f"calibration_quality=count:{trace.calibration_count}/"
26        f"{trace.minimum_calibration_count} "
27        f"bins:{trace.calibration_bin_count}/"
28        f"{trace.required_calibration_bin_count} "
29        f"brier:{trace.brier_score:.3f}/{trace.maximum_brier_score:.3f} "
30        f"gap:{trace.calibration_gap:.3f}/{trace.maximum_calibration_gap:.3f}"
31    )
32    print(f"tie_tolerance={trace.tie_tolerance:.12f}")
33    print(f"owner={trace.policy_owner} estimate={trace.estimate_id}")

Expected output

policy=refund-escalation-v1
event_probability=0.200
expected_costs=auto-clear:2.400,manual-review:2.000
decision=manual-review
brier=0.025
max_calibration_gap=0.150
scope=event:confirmed-abusive-refund population:authenticated-consumer-refunds horizon:30-days-after-request
versions=model:refund-risk-v3 calibration:temperature-2026-08-14 data:refund-outcomes-2026-07 evaluation:refund-calibration-eval-2026-08-14 costs:refund-costs-2026-08-14
evaluation_window=2026-07-01/2026-07-31
calibration_quality=count:4/4 bins:2/2 brier:0.025/0.050 gap:0.150/0.200
tie_tolerance=0.000000000001
owner=refund-risk-council estimate=estimate-1042

Verify: Run python3 -m unittest discover courses/ai-engineering/reference-impl/decision_probability.

01Resolve versionsRequire the estimate, calibration cohort, and policy to agree on event, population, horizon, model, calibrator, evaluation data, evaluation ID, window, and reliability-bin count; enforce the versioned minimum count, Brier, and calibration-gap gates before action selection.
02Compute every actionEmit the expected-cost vector, not only the winner, so operators can inspect margins and reproduce the choice.
03Make ambiguity executableVersion and bound tie tolerance inside the policy, then define tie, abstain, missing-feature, stale-calibration, and unavailable-review behavior instead of accepting a per-call override or language sort order.

Stress and operate the full decision loop

01Shift the base rateReplay a new population or time window. A stable ranking can coexist with stale probabilities and a now-wrong threshold.
02Move the cost tableVary missed-event harm, review cost, delay, and capacity. Report sensitivity ranges rather than presenting one uncertain estimate as fact.
03Create an exact cost tieVerify that the policy follows an explicit review or abstain rule instead of a container-order accident.
04Slice calibrationLook for high-confidence gaps in rare languages, new users, changed acquisition channels, and cases routed away from label collection.
05Poison numeric inputsReject bool, NaN, infinity, out-of-range probability, negative costs, and empty evidence before a decision is emitted.

Monitor	Signal	Response
probability quality	reliability with uncertainty by slice	recalibrate, narrow use, or roll back
policy distribution	action and abstention rates by version	inspect score, cost, or capacity shift
realized utility	outcome-conditioned cost and delay	revise cost estimates and policy
review operations	queue age, overrides, and coverage	throttle, prioritize, or add a safe fallback
label process	missingness, appeals, and feedback selection	repair measurement before trusting calibration

Operate at three altitudes

Production lens

— Version the event, population, horizon, model, calibrator, evaluation dataset/window, cost table, action set, policy owner, and estimate identity on every decision trace.
— Trace the policy-owned reliability-bin count plus observed calibration count, Brier score, and maximum gap beside their policy thresholds; reject an in-scope report that lacks adequate quality.
— Monitor discrimination, calibration, and realized decision utility separately by time and operational slice.
— Reject invalid, stale, or out-of-scope probabilities rather than coercing them into a default action.
— Include queue capacity, delayed labels, appeals, and selective observation in the operating model.

Staff lens

— Assign cost-table values to accountable domain and policy owners; they encode organizational choices, not model facts.
— Require threshold changes to name the probability, prevalence, cost, capacity, or authority assumption that changed.
— Design evaluation so model-influenced actions do not erase the counterfactual evidence needed to assess calibration and utility.

Interview defense

A classifier is 92% accurate. How would you choose the threshold for sending cases to manual review?

Accuracy does not determine the threshold. I would define the event and horizon, evaluate discrimination and calibration on independent time-appropriate data, estimate conditional probabilities for the deployment population, and enumerate allowed actions with outcome-conditioned costs including delay and review capacity. I would choose the minimum-expected-cost policy, specify tie and abstain behavior, test sensitivity to uncertain costs and prevalence, and monitor calibration, action rates, queue health, and realized utility by slice with model, calibrator, and policy versions attached.

Expect the interviewer to press on

— How does a base-rate shift affect the decision?
— Can a monotonic calibrator change ranking?
— What if review capacity is lower than the cost-optimal volume?
— How would selective labels bias the evaluation?

Misconceptions to remove

“A 0.8 model score means an 80% chance of correctness.”

Only a validated probability for a named event and population supports a frequency interpretation; many model scores are uncalibrated ranking values.

“The default 0.5 threshold is statistically neutral.”

It encodes a narrow cost and action assumption. Asymmetric consequences, constraints, or abstention options produce different policies.

“A lower Brier score proves better calibration.”

Brier score also reflects resolution and outcome uncertainty. Diagnose calibration with reliability evidence, uncertainty, and slices alongside proper scores.

Check your model

1. Why is P(evidence | event) insufficient to determine P(event | evidence)?

The reverse conditional also depends on the event prior and the overall evidence probability. Bayes' rule—or prior odds times a likelihood ratio—makes those dependencies explicit.

2. If action A has costs 12 for an event and 0 for no event, while action B always costs 2, when does B have lower expected cost?

A costs 12p and B costs 2, so B is lower-cost when p is greater than 1/6. At exactly 1/6 the policy needs an explicit tie rule.

3. Why can a model keep the same ROC-AUC while a production threshold becomes wrong?

Ranking can remain stable while prevalence or calibration changes, and the cost or capacity assumptions behind the threshold can change independently of the model.

Prove the mechanism

Add a third abstain action with a fixed delay cost, derive the probability ranges where each action wins, and test every exact boundary with an explicit tie policy.

Add a production constraint

Extend the artifact with weighted calibration observations and confidence intervals, then refuse automated action when the relevant slice lacks enough evidence for a declared calibration bound.

Artifact: Cost-sensitive decision policy

courses/ai-engineering/reference-impl/decision_probability/cost_sensitive_policy.py

Download reference implementation

Primary references and next links

References

1. On Calibration of Modern Neural Networks
Guo, Pleiss, Sun, and Weinberger. Primary ICML paper distinguishing confidence calibration from accuracy and evaluating post-hoc calibration methods.
2. Probability calibration
scikit-learn. Official guide defining calibrated probabilities, reliability diagrams, proper scores, and independent calibration data.
3. The Foundations of Cost-Sensitive Learning
Charles Elkan. Primary IJCAI paper grounding classifier decisions in explicit misclassification and example-dependent costs.

Continue through the graph

Define the Measurement Before the Model →
Connect event and outcome definitions to the decision evidence contract.
Data Contracts for Models →
Ensure probabilities are conditioned on valid, time-correct, declared inputs.
Academy roadmap →
Place probabilistic decision policy inside the broader learning-systems path.

Glossary: conditional probability · prior · likelihood ratio · posterior · calibration · Brier score · expected cost · decision threshold