Blob/Object Storage

You can enable blog/object storage for Pachyderm by updating the pachd.storage section in your Helm chart. The necessary configuration options depend on your chosen blob storage provider.

Before You Start #

• You should be familiar with the storageURL options in goCDK related to your blob storage provider.
  • AWS S3
  • Azure Blob
  • GCS Blob
• You should have the necessary credentials and permissions to create a bucket or container in your blob storage provider.

How to Set Up Blob Storage #

1. Navigate to your values.yaml file or obtain your current Helm values.yaml overrides:

   helm get values pachyderm  > values.yaml

   

2. Add the following pachd.storage fields to your values.yaml file:

   pachd:
     storage:
       gocdkEnabled: true 
       storageURL: "s3://my-bucket" or "gs://my-bucket" or "azure://my-container"

   

3. Update the storageURL to include provider-specific configuration options as needed; for options, see the related goCDK packages.

• AWS S3
• Azure Blob
• GCS Blob

4. Save your changes and upgrade your cluster:

   helm upgrade pachyderm pachyderm/pachyderm -f values.yaml

   

Limitations #

Some configuration settings such as verifySSL may not be passable via the storageURL as query parameters. In such cases, you can use the pachd.storage section to set these options.