🚧
Uploading and creating new Environments and models is restricted to beta users - please join the discord community to get access: Discord

To run your projects on the Mystic suite of software you need to use the python SDK, you can install this by running:

pip install pipeline-ai

Once installed you can authenticate with Catalyst or your PCore deployment by running

pipeline cluster login catalyst-api API_TOKEN -u https://www.mystic.ai -a

You can use this SDK to create "Pipelines" which can run locally or be uploaded and run remotely on Catalyst or your PCore deployment. You can remotely run the Pipeline via a standard REST API interface, or in the python SDK. We'll look at both of these before showing how to upload a Pipeline you've made.

To perform a basic inference request, known as a run, on a pre uploaded pipeline you can run the following for Llama 2 on Catalyst:

from pipeline.cloud.pipelines import run_pipeline
output = run_pipeline("meta/llama-2-7B:v7", "Hello my name is Paul, I like to", dict(max_new_tokens=20))

print(output.result.result_array())

The raw request for this is (put in your token):

curl --request POST \
  --url http://mystic.ai/v3/runs \
  --header 'Authorization: Bearer API_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{
	"pipeline_id_or_pointer": "meta/llama-2-7B:v7",
	"async_run": false,
	"input_data": [
		{
			"type": "string",
			"value": "Hellow my name is Paul, I like to"
		},
		{
			"type": "dictionary",
			"value": {
				"max_new_tokens": 20,
			}
		}
	]
}'

Uploading a Pipeline

There are a few basic concepts required for uploading a Pipeline:

Environment - What dependencies you need to remotely run your Pipeline
Accelerators - What hardware, GPU or CPU, do you need to be available when running your Pipeline

Environments

An Environment object contains a list of dependencies required to run your Pipeline. They're created via the python SDK, or API request, we recommend using the SDK as follows:

from pipeline.cloud import environments
from pipeline.configuration import current_configuration

# Turn on debbugging mode
current_configuration.set_debug_mode(True)

env_id = environments.create_environment("my_new_env", python_requirements=["numpy==1.25.2"])

When this is run an API request is sent to Catalyst or your PCore deployment to create this virtual environment. This environment is linked to future Pipelines when they're uploaded.

Creating a Pipeline

A Pipeline is a bit of code that describes how to tie together your code base to run and contain descriptions of what the input/outputs should be when executing. A simple example is given below:

from pipeline import Variable, pipe, Pipeline
from pipeline.configuration import current_configuration

# Turn on debbugging mode
current_configuration.set_debug_mode(True)

@pipe
def my_func(var: str) -> str:
    return f"Input string:'{var}'"

with Pipeline() as builder:
    input_1 = Variable(str)
    output_1 = my_func(input_1)
    builder.output(output_1)
  
my_pipeline = builder.get_pipeline()

my_pipeline.run("test")
# Input string:'test'

The full code to upload this to Catalyst or your PCore cluster is (this will by default run on CPUs):

from pipeline import Variable, pipe, Pipeline
from pipeline.cloud.pipelines import upload_pipeline, run_pipeline
from pipeline.configuration import current_configuration

# Turn on debbugging mode
current_configuration.set_debug_mode(True)

@pipe
def my_func(var: str) -> str:
    return f"Input string:'{var}'"

with Pipeline() as builder:
    input_1 = Variable(str)
    output_1 = my_func(input_1)
    builder.output(output_1)
  
my_pipeline = builder.get_pipeline()

new_pipeline = upload_pipeline(
    "<your-username>/helloworld", # The name for the pipeline, put your username here for catalyst (no username needed for pcore)
    environment_id_or_name=env_id, # The environment that we made earlier
)

output = run_pipeline(
  new_pipeline.id,
  "hello"
)
print(output.result.result_array())
# ["Input string :'test'"]