Multimodal Production Boundaries

Begin with a declared workload: this artifact models a service that requires both image and audio evidence. A text-only or image-only service would need a separately versioned modality policy, not a missing field interpreted as success. Bind model, evaluation, cohort, data, policy, ingestion, decode, privacy, license, moderation, source, and contract revisions. A result for another cohort or decoder version cannot silently substitute for the current release evidence.

These are independent obligations. A low error rate does not compensate for an unknown license state, and a fast response does not compensate for a missing modality. NIST's risk-management framing supports treating governance, measurement, and operational responsibility as contextual work rather than deriving trustworthiness from a single model score.

Compressed bytes do not determine decoded memory or model work. Images need pixel and dimension limits after decoding; audio needs duration, sampling, and channel policies; video adds frame and temporal sampling contracts. Enforce request counts and upload limits before expensive work, decode in an appropriately isolated component, then verify output bounds before allocating downstream tensors. The reference uses bounded hexadecimal placeholder payloads and declared pixel/duration values; it is not a real file parser.

A production decoder must attest that its observed dimensions and duration match the values submitted to the gate. Require explicit handling of corrupt data, orientation, color conversion, sample rates, and unsupported codecs. A declared decode=verified string is only a teaching stand-in for that evidence. No field name or content hash proves that a hostile file was actually decoded safely.

Privacy and rights checks belong before model consumption and before secondary retention or reuse. A payload can contain faces, speech, documents, background conversations, embedded metadata, or other sensitive information even when its filename looks harmless. The policy must say what is allowed, what is redacted, which processors receive the media, and what is retained. Unknown evidence is not equivalent to permission.

The local gate requires explicit verified states for privacy, licensing, and moderation. It also compares retention hours with the contract and requires deletion evidence. In a real system, deletion must include derived thumbnails, embeddings, caches, transcripts, backups where applicable, and any allowed replay records. Keep minimal identifiers and decision metadata for accountability without treating indefinite raw-media logging as necessary observability. Actual legal obligations depend on the deployment context and require the appropriate organizational review.

The artifact requires image OCR, image spatial, audio speech, and audio noise slices. Each slice binds the exact content digests for its modality, the shared model/evaluation/cohort scope, support, and error count. A missing slice blocks eligibility. A stale image digest in an OCR slice blocks eligibility. An aggregate score without these records cannot fill the gap. The four chosen slice names are an illustrative workload contract, not a universal multimodal benchmark.

MME and MMBench illustrate evaluation across multiple ability dimensions, rather than making every failure disappear into one number. For your service, derive slices from actual user tasks and harm modes: low-resolution documents, accents, background noise, language, chart types, or conflicting modalities may matter. Report uncertainty and error severity, not only a mean. A minimum support threshold is a gate against obvious absence of evidence, not a claim that the estimate is statistically conclusive.

The teaching gate checks missing modality, total media budget, decode, privacy, license, moderation, retention/deletion, slice coverage and content/quality, latency, then cost. Malformed schemas, duplicate identities, bad digests, and stale scopes raise errors before a policy report exists. Among well-formed records, the first failed boundary becomes the deterministic reason and the action is abstain. This ordering makes incident comparisons and replay tests stable; it does not imply later problems cease to exist.

Silently discarding an unavailable image or audio segment creates a different task. A text-only fallback is safe only if its limitations, user communication, privacy behavior, and quality have been separately evaluated. This artifact authorizes no such fallback; only abstention is accepted. If a production system introduces one, give it its own model, policy, evaluation, and telemetry identity rather than recycling the multimodal approval.

1def gate_release(contract: ReleaseContract, evidence: ReleaseEvidence) -> ReleaseReport:
2    """Fail closed in a fixed order; never release from aggregate-only evidence."""
3    validate_record(contract, ReleaseContract)
4    validate_record(evidence, ReleaseEvidence)
5    if any(item.scope != contract.scope for item in (evidence, *evidence.media, *evidence.slices)):
6        raise ValueError("ingestion/model/evaluation/cohort/policy scope mismatch")
7    if len({m.media_id for m in evidence.media}) != len(evidence.media) or len({(m.modality, m.payload_hex) for m in evidence.media}) != len(evidence.media):
8        raise ValueError("duplicate media identity or payload")
9    if len({s.slice_id for s in evidence.slices}) != len(evidence.slices):
10        raise ValueError("duplicate slice")

example=illustrative_only
status=ELIGIBLE_FOR_REVIEW
first_failure=none;action=review
claim=LOCAL_EVIDENCE_GATE_NOT_PRODUCTION_CERTIFICATION

The fixture supplies one tiny placeholder image record and one audio record, four supported slices, and declared handling and serving evidence. Frozen records defensively normalize sequences and bind all declarations with content digests. Every public call reconstructs nested records to catch constructor bypass, stale identities, boolean counts, NaN, infinity, subnormal values, invalid ranges, and missing fields. Duplicate media IDs, duplicate payloads, or duplicate slice names cannot inflate evidence.

ELIGIBLE_FOR_REVIEW means only that these local declarations meet this local policy. The script does not call a decoder, inspect licenses, moderate content, confirm deletion, measure latency, run evaluations, or release software. Those measurements and attestations must come from trusted external processes. The artifact makes their binding and decision rules inspectable; it cannot manufacture their truth.

Collect failure reasons by model and policy cohort, and distinguish ingestion rejection, moderation abstention, missing-slice blocks, latency budget failures, and user cancellations. Track latency distributions across decode, encoder, prefill, and generation stages rather than treating a single average as capacity evidence. Keep cost attribution tied to the same request shape and modality mix. A quieter error dashboard after suppressing a modality is not an improvement if the service no longer performs the promised task.

Stress a release with a valid aggregate but missing slices, an unknown privacy state, stale evaluation media, a decoder failure combined with a cost overrun, and a deletion job that failed after successful inference. Rehearse rollback of the model, preprocessing, and policies together. A prior checkpoint with a new incompatible ingestion path is not necessarily the previously approved system. Stop at the first safe boundary, preserve minimal replay evidence, and assign remediation to the responsible owner.