GPUs and accelerators

When it comes time to deploy your Pipeline, there are a few extra infrastructure considerations. The pipeline.yaml file provides a way to specify what type of hardware your Pipeline will need to run on (this is usually something like an NVIDIA graphics card).

Concepts

  • Accelerator - The piece of hardware to run with your Pipeline
  • VRAM - The estimated amount of VRAM your Pipeline will use on a GPU

Specify your hardware requirements in your YAML file like so:

accelerators: ["nvidia_a100", "nvidia_a100"]
accelerator_memory: 60000

Notice how you can specify multiple GPUs, just make sure your specification is one of the available configurations listed below!

If accelerators is an empty list, it will default to a "cpu" configuration.

Accelerator list

The accelerators that we currently offer are:

NameMax VRAM
nvidia_t416GB
nvidia_a10040GB
nvidia_a100_80gb80GB
nvidia_v10016GB
nvidia_v100_32gb32GB
nvidia_309024GB
nvidia_a1616GB
nvidia_h10080GB
nvidia_l424GB
nvidia_all<10GB
cpu-

Catalyst Accelerator configurations

You can configure the above accelerators in any of the configurations below:

AcceleratorsPer second costPer hour cost
1x Nvidia T4$0.0000694$0.25
1x Nvidia A100 (40GB)$0.000556$2
2x Nvidia A100 (40GB)$0.00139$5
4x Nvidia A100 (40GB)$0.00278$10
1x Nvidia A100 (80GB)$0.000694$2.50
2x Nvidia A100 (80GB)$0.00167$6
4x Nvidia A100 (80GB)$0.00333$12
1x Nvidia V100 (16GB)$0.000347$1.25
1x Nvidia V100 (32GB)$0.000417$1.50
1x Nvidia L4$0.000138$0.50
1x Nvidia A5000$0.000138$0.50
Nvidia all*$0.000278$1
CPU$0.0000278$0.10

*When using the "Nvidia all" option a random GPU is selected if enough VRAM is required