Virtual environments

Both Catalyst and pcore clusters use virtual python environments (venvs) to run all pipelines. Once a venv is created it can be linked with a pipeline when uploaded. You can create as many venvs as you like and can attach one venv to multiple pipelines.


  • Requirements - python packages to be included in your virtual env

Environment creation

To create a new venv on Catalyst or a pcore cluster you import the create function from the pipeline-ai library (replace <username> with your username):

from import create_environment

env_id = create_environment(name="<username>/numpy", python_requirements=["numpy==1.25.2"])

This environment can then be tagged with a pipeline when uploading like so:

from import upload_pipeline



We highly reccomend pointing to specific package versions as defined on


Fixed requirements

Once an environment is created its requirements cannot be changed, you must create a new one if you need to modify the requirements.

Git repos

It is possible to pass in a specific git repo as a package, we strongly recommend pointing to a specific commit as follows:

from import create_environment

env_id = create_environment(

Current limitations

  • We do not currently support OS level packages, no apt install but will be coming in a future release. The only package you can expect at the moment is ffmpeg - please reach out to the team on discord if you have an urgent need for an OS level package: Discord.
  • Only python==3.10 venvs are currently supported, and you must upload new pipelines from python==3.10.
  • The only supported source language is python for uploading and running the venv.