When it comes time to deploy your Pipeline, there are a few extra infrastructure considerations. The pipeline.yaml file provides a way to specify what type of hardware your Pipeline will need to run on (this is usually something like an NVIDIA graphics card).

Concepts

• Accelerator - The piece of hardware to run with your Pipeline
• VRAM - The estimated amount of VRAM your Pipeline will use on a GPU (in MB)

Specify your hardware requirements in your YAML file like so:

accelerators: ["nvidia_a100", "nvidia_a100"]
accelerator_memory: 60000

Notice how you can specify multiple GPUs, just make sure your specification is one of the available configurations listed below!

If accelerators is an empty list, it will default to a "cpu" configuration.

Accelerator list

The accelerators that we currently offer are:

*Fractional GPU
**BYOC Only

Fractional GPUs and GPU sharing

Mystic does offer fractional GPUs! At the moment they're limited to the A100 GPUs but will be supported with H100s in an upcoming release.

Supported fractions:

You define these as normal in the accelerator field in the pipeline.yaml with your project:

runtime:
  container_commands:
    ...
  python:
    ...
accelerators: ["nvidia_a100_5gb"]
pipeline_graph: new_pipeline:my_new_pipeline
pipeline_name: paulh/a-cool-model
readme: README.md
...

Fractional pricing

👍

No low volume premium for fractional GPUs

There is a completely flat pricing for fractional GPUs, there is not a premium for the first 10hours!

Catalyst Accelerator configurations

You can configure the above accelerators in any of the configurations below:

*When using the "Nvidia all" option a random GPU is selected if enough VRAM is required.