Getting Started

Welcome! This guide contains a handful of simple examples to help you get started with running batch jobs, deploying services, and orchestrating complex workflows in Covalent Cloud. You'll need only basic Python knowledge to follow along — no prior cloud, Kubernetes, or Docker experience is required!

Installation and Initial Setup

Start by installing the Covalent Cloud SDK. We recommend using a local Python environment with Python version 3.9 to 3.11.

pip install -U covalent-cloud

Create an account then retrieve your API key from the dashboard and save it.

import covalent_cloud as cc

cc.save_api_key("<your-API-key-here>")

Environment Setup

Skip the hassle of managing container images. With Covalent Cloud, you can directly create, delete, and modify reusable cloud environments.

cc.create_env(
    name="sklearn-env",
    pip=["scikit-learn==1.5.2", "yfinance==0.2.48", "numpy==1.23.5"],
    wait=True,
)

See our Environment Management Guide for more on cloud environments.

Define Compute Resources

Next, let's create an executor to use in this guide. In Covalent Cloud, executors specify a modular set of compute resources for tasks and services.

cpu_ex = cc.CloudExecutor(
    env="sklearn-env",
    num_cpus=2,
    memory="8GB",
    time_limit="2 hours"
)

This guide does not require them, but to use GPUs we'd simply specify the gpu_type and num_gpus:

gpu_ex = cc.CloudExecutor(
    env="sklearn-env",
    num_cpus=24,
    num_gpus=4,
    gpu_type="h100",
    time_limit="30 minutes"
)

Making a CloudExecutor instance does not provision or reserve anything in the cloud. You can think of executors as specifications for the resources you want to use. Any executor can be assigned to any task or service. Covalent Cloud only creates resources when a task is executed or service is deployed.

See our Compute Resources Guide for more information.

• Compute Jobs
• Inference API endpoints
• Orchestrating Compute Workflows

Running batch jobs or tasks in Covalent Cloud is straightforward. Creating tasks is as simple as adding a single decorator to almost any Python function. In the example below, we train a stock predictor using the environment and compute resources defined above.

As written, this example requires yfinance and scikit-learn to be installed in your local environment.

pip install yfinance==0.2.48 scikit-learn==1.5.2

Local requirements can generally be avoided by simply moving relevant imports inside the decorated function.

import covalent as ct

import sklearn
import sklearn.svm
import yfinance as yf
from sklearn.model_selection import train_test_split


# One-task workflow when stacking decorators:
@ct.lattice(executor=cpu_ex, workflow_executor=cpu_ex)
@ct.electron(executor=cpu_ex)
def fit_svr_model_and_evaluate(ticker, n_chunks, C=1):

    ticker_data = yf.download(ticker, start='2022-01-01', end='2023-01-01')
    data = ticker_data.Close.to_numpy()
    X = [data[i:i+n_chunks].squeeze() for i in range(len(data) - n_chunks)]
    y = data[n_chunks:].squeeze()

    # Split data into train and test sets
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.2, shuffle=False)

    # Fit SVR model
    model = sklearn.svm.SVR(C=C).fit(X_train, y_train)

    # Predict and calculate MSE
    predictions = model.predict(X_test)
    mse = sklearn.metrics.root_mean_squared_error(y_test, predictions)

    return model, mse


# Run the one-task workflow
runid = cc.dispatch(fit_svr_model_and_evaluate)('AAPL', n_chunks=6, C=10)
model, mse = cc.get_result(runid, wait=True).result.load()

print(model, mse)

Full Code

import covalent as ct
import covalent_cloud as cc

import sklearn
import sklearn.svm
import yfinance as yf
from sklearn.model_selection import train_test_split

cc.save_api_key("<your-API-key-here>")


cc.create_env(
    name="sklearn-env",
    pip=["scikit-learn==1.5.2", "yfinance==0.2.48", "numpy==1.23.5"],
    wait=True,
)

cpu_ex = cc.CloudExecutor(
    env="sklearn-env",
    num_cpus=2,
    memory="8GB",
    time_limit="2 hours"
)


# One-task workflow when stacking decorators:
@ct.lattice(executor=cpu_ex, workflow_executor=cpu_ex)
@ct.electron(executor=cpu_ex)
def fit_svr_model_and_evaluate(ticker, n_chunks, C=1):

    ticker_data = yf.download(ticker, start='2022-01-01', end='2023-01-01')
    data = ticker_data.Close.to_numpy()
    X = [data[i:i+n_chunks].squeeze() for i in range(len(data) - n_chunks)]
    y = data[n_chunks:].squeeze()

    # Split data into train and test sets
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.2, shuffle=False)

    # Fit SVR model
    model = sklearn.svm.SVR(C=C).fit(X_train, y_train)

    # Predict and calculate MSE
    predictions = model.predict(X_test)
    mse = sklearn.metrics.root_mean_squared_error(y_test, predictions)

    return model, mse


# Run the one-task workflow
runid = cc.dispatch(fit_svr_model_and_evaluate)('AAPL', n_chunks=6, C=10)
model, mse = cc.get_result(runid, wait=True).result.load()

print(model, mse)