Blackwell (B200, B300) and Hopper (H100, H200) are two generations of NVIDIA data center GPUs. Blackwell is newer and has more memory per GPU. Halo supports both, with a separate image for each.
The two generations
Hopper is the NVIDIA GPU generation of the H100 and H200. Blackwell is the one after it, with the B200 and B300. The B200 and B300 have more memory than the H100 and H200, so the same model needs less sharding and a longer sequence fits.
Which to pick
- Blackwell when memory is the limit: a large mixture-of-experts model, or long sequences. Fewer GPUs hold the same model. See how many GPUs you need.
- Hopper when that is what your cluster has. A model that does not fit on one GPU is sharded across more of them.
Compare throughput numbers only within a generation. A number measured on one does not carry over to the other.
In Halo
Use the image that matches the GPU. Each carries the kernels built for its generation:
Bash
docker pull public.ecr.aws/whitecircle/halo:blackwell # B200/B300
docker pull public.ecr.aws/whitecircle/halo:hopper # H100/H200Halo picks Flash Attention 4 on Blackwell and Flash Attention 3 on Hopper. There is no latest tag, so a Hopper host cannot pull a Blackwell image by mistake. Both images are x86_64-only, so Arm-based GB200 and GB300 hosts have no image. Halo's published throughput numbers were measured on B300 GPUs. See installation.
