Schedule heterogeneous accelerator workloads with fairness and preemption

• Classify latency, gang size, topology, checkpoint cost, deadline, and accelerator compatibility. Define organizational fairness: quotas, priorities, reservations, and whether unused capacity may be borrowed.

• Model GPUs by shape and interconnect, queue arrival distributions, job duration, checkpoint interval, and regional inventory. Raw utilization hides fragmentation and unusable partial allocations.

• Protect inference floors, schedule gang jobs atomically, minimize fragmentation, support safe preemption, expose queue expectations, and account usage to teams.

• A global control plane accepts workload specs and quota policy; regional schedulers own current inventory and placements. Use hierarchical fair queues, reservations, backfill of short jobs, topology-aware gang scheduling, and checkpoint-aware preemption.

• Workloads declare resource shape, priority, deadline, estimated duration, checkpoint location, and tenant. Leases fence placements; heartbeats and checkpoint events drive reclaim and reschedule.

• Resource allocation uses fenced leases so stale agents cannot run duplicate exclusive jobs. Usage accounting is idempotent and eventually reconciled from allocation and device telemetry.

• On node failure, restart from verified checkpoint; on scheduler partition, existing leases run until bounded expiry while new allocation stops locally. Preserve a control lane and inference reserve.

• Isolate device memory and storage, authenticate artifacts and jobs, and prevent one tenant from reading checkpoints or topology details. Privileged drivers and schedulers have tightly scoped roles.

• Measure useful accelerator time, fragmentation, queue delay by class, preemption waste, checkpoint age, failed gang placement, quota borrowing, and inference SLO.

• Use commitments for stable inference, spot or interruptible capacity for checkpointable batch, and shape-aware demand forecasts. Optimize completed business work per accelerator-dollar, not utilization alone.

• Preemption improves priority response but destroys uncheckpointed work; strict reservations protect SLOs but strand capacity. Borrow idle reservations with revocation tied to checkpointability and deadline.

• Begin with accurate inventory and usage attribution, then quotas, reservations, backfill, and preemption. Introduce automatic checkpoint policy before allowing aggressive revocation.

• Principal answers join scheduling algorithm, workload contract, team policy, and hardware economics. They call out fragmentation, fencing, and productive utilization.

Interviewer evaluation rubric