The Model-Serving Control Plane

A useful boundary is temporal as well as architectural. The data plane handles work whose correctness and latency are visible to callers: admission, routing, batching, model execution, streaming, cancellation, and request telemetry. The control plane handles slower convergence: revision registration, placement, artifact loading, desired replica counts, health policy, autoscaling intent, traffic stages, and rollback. The request path consumes an already-authorized routing state; it should not synchronously decide placements or fetch a mutable model alias.

A reconciler repeatedly reads desired and observed state, validates scope and freshness, computes a next decision, and records it with a content identity. The operation must be idempotent: replaying the same canonical evidence yields the same answer and does not double-advance traffic. Separate readiness evidence from performance regression evidence because they imply different actions. Missing readiness holds the current fraction; a demonstrated request regression rolls back to the previous revision.

1. 01Resolve identity and authorityBind model, configuration, placement, health, autoscaling, rollback, and fixture or environment scopes before comparing any observations.
2. 02Reconcile desired allocationRequire the target and rollback revisions to reach their declared desired state, and reserve surge capacity from desired allocations rather than transient ready counts.
3. 03Qualify model loading and healthUse bounded attempts, success rates, fresh health, and representative inference evidence; do not equate process liveness with a correctly loaded model.
4. 04Decide exactly one transitionRollback on hard request regressions, hold on incomplete or ceiling-bound evidence, and otherwise advance to one capped traffic fraction.

Deployment, autoscaling, placement, model loading, and traffic controllers can each be locally correct while their combined loop oscillates. Define who owns each field, the measurement window, stabilization and cooldown behavior, minimum capacity per rollout stage, and precedence during rollback. The autoscaler should not erase canary capacity, and rollout surge should not consume rollback headroom. A health signal should identify the revision and remain fresh across the same observation windows used by the traffic decision.

The reference artifact reconstructs exact frozen records, verifies their content digests and shared scope, and returns one local decision. It rejects booleans masquerading as counts, string subclasses, non-finite floats, constructor-bypassed records, stale contract bindings, desired-capacity overcommit, incomplete rollback state, and traffic above the policy boundary. It never contacts a cluster, changes routing, or claims deployment success.

1def plan_rollout(contract: RolloutContract, evidence: RolloutEvidence) -> RolloutPlan:
2    contract = validate_record(contract, RolloutContract)
3    evidence = validate_record(evidence, RolloutEvidence)
4    if evidence.scope != contract.scope or evidence.contract_content_id != contract.content_id:
5        raise ValueError("evidence belongs to another rollout contract")
6 
7    # Reconcile identity, desired allocation, readiness, rollback, and request gates.
8    # Regressions roll back; incomplete or ceiling-bound evidence holds.
9    # Only fully qualified evidence advances one capped traffic stage.

example=illustrative_only
decision=ADVANCE_BOUNDED_ROLLOUT
target_revision=model-v18;previous_revision=model-v17
target_ready=4/4;load_success_rate=1.000
error_rate=0.002;p95_latency_ms=180.000
next_traffic_fraction=0.200
claim=LOCAL_CONTROL_PLANE_AUDIT_NOT_DEPLOYMENT

A model repository is executable-system input, not a passive file share. Restrict who can publish revisions, verify artifact and configuration digests before loading, separate tenant namespaces, make provenance auditable, and minimize dynamic repository mutation. Triton's documentation warns that dynamic model loading can expose arbitrary-code execution risk when a repository is writable by an untrusted party; disable or tightly authorize repository control when the workflow does not require it.

• Reject target evidence whose model, configuration, scope, or policy identity differs from the approved revision.
• Expire health and load observations so a once-healthy replica cannot authorize a later rollout after drift or restart.
• Protect routing and rollback commands with least privilege, authenticated provenance, replay resistance, and an append-only decision trail.
• Exercise corrupt artifacts, partial downloads, incompatible runtimes, slow loads, poisoned health, unavailable registries, controller restarts, and concurrent reconciliation.
• Keep the previous revision warm enough to recover, and prove rollback routing rather than inferring it from object existence.

Persist each desired state, canonical observation set, decision, actor or controller identity, and resulting state in a transition ledger. Join data-plane metrics by model and configuration revision, rollout stage, placement, and policy scope. Alert on reconciliation age, repeated holds, desired-ready gaps, load failures, stale health, controller write conflicts, rollback-route loss, surge exhaustion, and request regressions. Rehearse controller restart and duplicated events: recovery should reconstruct state from durable records rather than from an in-memory step counter.

1. 01Qualify before exposureVerify immutable identity, placement, desired readiness, load success, representative health, rollback route, and capacity before moving traffic.
2. 02Observe through complete windowsUse minimum sample and time windows that cover warmup, representative request shapes, autoscaler reaction, and downstream behavior.
3. 03Stop on ambiguous ownershipIf multiple controllers can write replicas, routing, or revision aliases without precedence, hold and repair the authority model.
4. 04Close the rollout deliberatelyDistinguish a bounded policy ceiling from completion, then require a separately authorized promotion or terminal state before retiring rollback capacity.