Posts

A GB200-class GPU cluster emulated on a single Linux machine: what it is, what you can practise with it, and how to set it up.

Running the emulated GB200 cluster with Slurm: GPU scheduling, jobs from srun to PyTorch DDP, drains, quotas and monitoring.

The same emulated GPU cluster on Kubernetes: GPU pods, Kueue topology-aware queueing, JobSet and monitoring.

Out-of-band management of GPU trays with Redfish BMCs: inventory, GPU sensors, NVLink faults and power cycles.

NVLink partitions, fabric health and telemetry, and how a partition change reaches the scheduler.

What to monitor on a GPU cluster, layer by layer, and how to read the graphs, from GPUs to BMCs.