GPU Sharing on Kubernetes: Time-Slicing vs MIG vs MPS

💡 Quick Answer: Kubernetes can’t split nvidia.com/gpu natively; the NVIDIA device plugin (via the GPU Operator) offers three ways to share one GPU. Time-slicing — any NVIDIA GPU, pods take turns, no memory or fault isolation; good for notebooks/dev. MPS — processes run concurrently with per-client memory and compute limits, but a fault can still affect other clients; good for many small inference pods. MIG — hardware partitions with dedicated memory, cache and SMs and full fault isolation, on MIG-capable data-center GPUs (A30, A100, H100/H200, Blackwell); good for multi-tenant production inference. Keep full GPUs for training.

Comparison

Overhead: MIG and MPS add little; time-slicing costs context switches and, more importantly, every pod sees contention from the others. Measure with your workload rather than trusting a percentage.

graph TD
    A[GPU workload] --> B{Type?}
    B -->|Training / large model| C[Full GPU]
    B -->|Inference| D{Need memory + fault isolation?}
    B -->|Notebook / dev / CI| E[Time-slicing]
    D -->|Yes, multi-tenant / SLA| F[MIG]
    D -->|No, trusted pods, want concurrency| G[MPS]
    D -->|GPU not MIG-capable| G

Time-Slicing

Device plugin config in a ConfigMap, referenced from the ClusterPolicy:

apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
  namespace: gpu-operator
data:
  any: |-
    version: v1
    sharing:
      timeSlicing:
        renameByDefault: false
        failRequestsGreaterThanOne: true
        resources:
          - name: nvidia.com/gpu
            replicas: 4

kubectl patch clusterpolicies.nvidia.com/cluster-policy --type merge \
  -p '{"spec":{"devicePlugin":{"config":{"name":"time-slicing-config","default":"any"}}}}'

Each GPU is then advertised as 4 nvidia.com/gpu. A pod requesting nvidia.com/gpu: 1 gets a share, not a quarter — nothing stops it from using all the memory. Full walkthrough, per-node profiles and verification: GPU time-slicing on Kubernetes.

MPS

Supported by the NVIDIA device plugin since v0.15 (initially flagged experimental; shipped with recent GPU Operator releases). The plugin runs an MPS control daemon per GPU and splits it into equal replicas:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mps-config
  namespace: gpu-operator
data:
  mps-4: |-
    version: v1
    sharing:
      mps:
        resources:
          - name: nvidia.com/gpu
            replicas: 4

With replicas: 4, each client is limited to roughly a quarter of the GPU’s memory and compute. Apply it like time-slicing (ClusterPolicy devicePlugin.config, optionally per node with the nvidia.com/device-plugin.config=mps-4 label). The device plugin does not support MPS on MIG-enabled GPUs.

MIG

The GPU Operator’s MIG manager partitions GPUs according to the node label nvidia.com/mig.config. The built-in default-mig-parted-config already contains common layouts (all-1g.10gb, all-3g.40gb, all-balanced, …):

# choose the strategy once (mixed = per-profile resource names)
kubectl patch clusterpolicies.nvidia.com/cluster-policy --type merge \
  -p '{"spec":{"mig":{"strategy":"mixed"}}}'

# partition a node (A100/H100 80GB: 7 × 1g.10gb)
kubectl label node gpu-node-1 nvidia.com/mig.config=all-1g.10gb --overwrite

# watch the MIG manager apply it
kubectl get node gpu-node-1 -L nvidia.com/mig.config.state

For custom layouts, point spec.migManager.config.name at your own ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: custom-mig-config
  namespace: gpu-operator
data:
  config.yaml: |
    version: v1
    mig-configs:
      one-big-three-small:          # A100/H100 80GB
        - devices: all
          mig-enabled: true
          mig-devices:
            "3g.40gb": 1
            "1g.10gb": 3

Request a slice by profile (mixed strategy):

resources:
  limits:
    nvidia.com/mig-1g.10gb: 1

Profile names depend on the GPU’s memory size:

Run nvidia-smi mig -lgip on the node for the authoritative list. With strategy: single all GPUs on a node use one profile and slices appear as plain nvidia.com/gpu.

Repartitioning needs the GPU idle: the MIG manager stops GPU operand pods on the node, and workloads using the GPU must be drained first. On some platforms enabling MIG mode also requires a GPU reset or node reboot. Platform-specific procedures: OpenShift MIG reconfiguration, Talos + MIG, and dynamic partitioning with DRA.

Combining Strategies

• Per node: label nodes for their role (full GPUs for training, MIG for inference, time-slicing for notebooks) and steer workloads with nodeSelector/affinity. Don’t mix strategies on the same GPU unless you mean to.
• Time-slicing on top of MIG: the time-slicing config can oversubscribe MIG devices too (e.g. name: nvidia.com/mig-1g.10gb, replicas: 2) — isolation between slices, sharing within one.
• Scheduler-level sharing: fractional GPU requests and fair-share queues are what KAI Scheduler adds on top.
• Quotas: cap GPU consumption per namespace with ResourceQuotas; note that a time-sliced or MPS replica counts as one nvidia.com/gpu.

Common Issues

Frequently Asked Questions

What is the difference between GPU time-slicing and MIG?

Time-slicing lets several pods take turns on the whole GPU; they share all memory and there is no fault isolation, but it works on any NVIDIA GPU. MIG splits a supported GPU into hardware partitions, each with its own memory, cache and SMs, so one tenant can’t exhaust another’s memory or crash its work.

When should I use MPS instead of time-slicing?

When several small, trusted inference processes should run concurrently on one GPU with per-client memory and compute limits. MPS gives better utilisation than time-slicing for small kernels, but lacks MIG’s fault isolation.

Which GPUs support MIG?

Data-center GPUs from Ampere onward that NVIDIA lists as MIG-capable: A30 (up to 4 instances), A100, H100, H200, and Blackwell B200/GB200 (up to 7), plus some RTX PRO Blackwell cards. T4, V100, L4 and L40S do not support MIG.

Can I combine MIG and time-slicing?

Yes. Configure time-slicing for a MIG resource name (for example nvidia.com/mig-1g.10gb with replicas: 2) to oversubscribe each slice while keeping hardware isolation between slices.

Does time-slicing limit GPU memory per pod?

No. Every time-sliced pod can allocate the full GPU memory. Use MPS or MIG for enforced limits, or cap allocation inside the application.