Skip to main content

AWS Plugins

Covalent is a python based workflow orchestration tool used to execute HPC and quantum tasks in heterogenous environments.

By installing Covalent AWS Plugins users can leverage a broad plugin ecosystem to execute tasks using AWS resources best fit for each task.

Covalent AWS Plugins installs a set of executor plugins that allow tasks to be run in an EC2 instance, AWS Lambda, AWS ECS Cluster, AWS Batch Compute Environment, and as an AWS Braket Job for tasks requiring Quantum devices.

If you’re new to covalent visit our Getting Started Guide.

1. Installation

To use the AWS plugin ecosystem with Covalent, simply install it with pip:

pip install "covalent-aws-plugins[all]"

This will ensure that all the AWS executor plugins listed below are installed.


Users will require Terraform to be installed in order to use the EC2 plugin.

2. Included Plugins

While each plugin can be separately installed installing the above pip package installs all of the below plugins.

PluginsPlugin NameUse Case
AWS Batch Executor Useful for heavy compute workloads (high CPU/memory). Tasks are queued to execute in the user defined Batch compute environment.
AWS EC2 ExecutorGeneral purpose compute workloads where users can select compute resources. An EC2 instance is auto-provisioned using Terraform with selected compute settings to execute tasks.
AWS Braket ExecutorSuitable for Quantum/Classical hybrid workflows. Tasks are executed using a combination of classical and quantum devices.
AWS ECS Executor Useful for moderate to heavy workloads (low memory requirements). Tasks are executed in an AWS ECS cluster as containers.
AWS Lambda Executor Suitable for short lived tasks that can be parallalized (low memory requirements). Tasks are executed in serverless AWS Lambda functions.

3. Usage Example

  • Firstly, import covalent
import covalent as ct
  • Secondly, define your executor
executor = ct.executor.AWSBatchExecutor(
s3_bucket_name = "covalent-batch-qa-job-resources",
batch_job_definition_name = "covalent-batch-qa-job-definition",
batch_queue = "covalent-batch-qa-queue",
batch_execution_role_name = "ecsTaskExecutionRole",
batch_job_role_name = "covalent-batch-qa-job-role",
batch_job_log_group_name = "covalent-batch-qa-log-group",
vcpu = 2, # Number of vCPUs to allocate
memory = 3.75, # Memory in GB to allocate
time_limit = 300, # Time limit of job in seconds
  • Lastly, define a workflow to execute a particular task using one of the above executors
def compute_pi(n):
# Leibniz formula for π
return 4 * sum(1.0/(2*i + 1)*(-1)**i for i in range(n))

def workflow(n):
return compute_pi(n)

dispatch_id = ct.dispatch(workflow)(100000000)
result = ct.get_result(dispatch_id=dispatch_id, wait=True)

Which should output