GPUs and Accelerators

Concepts

  • Accelerator - The piece of hardware to run with your Pipeline
  • VRAM - The estimated amount of VRAM your Pipeline will use on a GPU, entered in the required_gpu_vram_mb field on upload (this will be deprecated in a future release).

An Accelerator is coupled with a Pipeline at upload time by using the accelerators field in upload_pipeline as shown below:

from pipeline.cloud.compute_requirements import Accelerator
from pipeline.cloud.pipelines import upload_pipeline
...
remote_pipeline = upload_pipeline(
  my_pipeline,
  "paulh/new-pipeline"
  "default-env",
  required_gpu_vram_mb=30_000,
  accelerators=[
    Accelerator.nvidia_a100,
  ]
)

Accelerator list

You can access all Accelerators in the Python SDK by importing them from the compute_requirements module:

from pipeline.cloud.compute_requirements import Accelerator
NameMax VRAM
Accelerator.nvidia_t416GB
Accelerator.nvidia_a10040GB
Accelerator.nvidia_a100_80gb80GB
Accelerator.nvidia_v10016GB
Accelerator.nvidia__32gb32GB
Accelerator.nvidia_309024GB
Accelerator.nvidia_a1616GB
Accelerator.nvidia_h10080GB
Accelerator.nvidia_l424GB
Accelerator.nvidia_all<10GB
Accelerator.cpu-

Catalyst Accelerator configurations

AcceleratorsPer second costPer hour cost
1x Nvidia T4$0.0000694$0.25
1x Nvidia A100 (40GB)$0.000556$2
2x Nvidia A100 (40GB)$0.00139$5
4x Nvidia A100 (40GB)$0.00278$10
1x Nvidia A100 (80GB)$0.000694$2.50
2x Nvidia A100 (80GB)$0.00167$6
4x Nvidia A100 (80GB)$0.00333$12
1x Nvidia V100 (16GB)$0.000347$1.25
1x Nvidia V100 (32GB)$0.000417$1.50
1x Nvidia L4$0.000138$0.50
1x Nvidia A5000$0.000138$0.50
Nvidia all*$0.000278$1
CPU$0.0000278$0.10

*When using the "Nvidia all" option a random GPU is selected if enough VRAM is required

Multiple GPUs

To use any of the multi GPU configurations you simply append more of the accelerators in the upload_pipeline accelerators field:

from pipeline.cloud.compute_requirements import Accelerator
from pipeline.cloud.pipelines import upload_pipeline
...
remote_pipeline = upload_pipeline(
  my_pipeline,
  "paulh/new-pipeline"
  "default-env",
  required_gpu_vram_mb=50_000,
  accelerators=[
    Accelerator.nvidia_a100,
    Accelerator.nvidia_a100,
  ]
)