Target Audience

Intro to Data Versioning

Intro to Pipelines

👉 Get Started

Local Getting Started Guides

Cloud Getting Started Guides

Azure Deployment

Install PachCTL Auto-completion

Beginner Tutorial

Advanced Data Operations

Analyze Time-Windowed Data

Mount a Volume in a Pipeline

Skip Failed Datums

Basic Data Operations

Egress To An SQL Database

Export via Egress

Export via PachCTL

Mount a Repo Locally

Manage Commits & Delete Data

Debugging Operations

View Audit Logs

View Kubernetes Logs

Developer Workflow

CI/CD Integration

Create a Machine Learning Workflow

The Push Images Flag

Working with Pipelines

JupyterLab Mount Extension

Docker Installation Guide

Local Installation Guide

Troubleshooting

Pipeline Operations

Create a Pipeline

Delete a Pipeline

Jsonnet Pipeline Specifications

Update a Pipeline

Project Operations

Create a Project

Set a Project as Current

Add a Project Resource

Grant Project Access

Delete a Project

S3 Gateway Operations

Create S3 Bucket

Delete an S3 Object

Delete Empty S3 Bucket

Get an S3 Object

List S3 Buckets

List S3 Objects

Write an S3 Object

Pachctl auth activate)

Pachctl auth check)

Pachctl auth check project)

Pachctl auth check repo)

Pachctl auth deactivate)

Pachctl auth get config)

Pachctl auth get groups)

Pachctl auth get robot token)

Pachctl auth get)

Pachctl auth get cluster)

Pachctl auth get enterprise)

Pachctl auth get project)

Pachctl auth get repo)

Pachctl auth login)

Pachctl auth logout)

Pachctl auth revoke)

Pachctl auth roles for permission)

Pachctl auth rotate root token)

Pachctl auth set config)

Pachctl auth set)

Pachctl auth set cluster)

Pachctl auth set enterprise)

Pachctl auth set project)

Pachctl auth set repo)

Pachctl auth use auth token)

Pachctl auth whoami)

Pachctl buildinfo)

Pachctl completion)

Pachctl completion bash)

Pachctl completion zsh)

Pachctl config)

Pachctl config delete)

Pachctl config delete context)

Pachctl config get)

Pachctl config get active context)

Pachctl config get active enterprise context)

Pachctl config get context)

Pachctl config get metrics)

Pachctl config import kube)

Pachctl config list)

Pachctl config list context)

Pachctl config set)

Pachctl config set active context)

Pachctl config set active enterprise context)

Pachctl config set context)

Pachctl config set metrics)

Pachctl config update)

Pachctl config update context)

Pachctl connect)

Pachctl copy file)

Pachctl create)

Pachctl create branch)

Pachctl create pipeline)

Pachctl create project)

Pachctl create repo)

Pachctl create secret)

Pachctl debug analyze)

Pachctl debug binary)

Pachctl debug dump)

Pachctl debug profile)

Pachctl delete)

Pachctl delete all)

Pachctl delete branch)

Pachctl delete commit)

Pachctl delete file)

Pachctl delete job)

Pachctl delete pipeline)

Pachctl delete project)

Pachctl delete repo)

Pachctl delete secret)

Pachctl delete transaction)

Pachctl diff file)

Pachctl draw pipeline)

Pachctl edit pipeline)

Pachctl enterprise)

Pachctl enterprise deactivate)

Pachctl enterprise get state)

Pachctl enterprise heartbeat)

Pachctl enterprise pause status)

Pachctl enterprise pause)

Pachctl enterprise register)

Pachctl enterprise sync contexts)

Pachctl enterprise unpause)

Pachctl finish)

Pachctl finish commit)

Pachctl finish transaction)

Pachctl get file)

Pachctl glob file)

Pachctl idp create client)

Pachctl idp create connector)

Pachctl idp delete client)

Pachctl idp delete connector)

Pachctl idp get client)

Pachctl idp get config)

Pachctl idp get connector)

Pachctl idp list client)

Pachctl idp list connector)

Pachctl idp set config)

Pachctl idp update client)

Pachctl idp update connector)

Pachctl inspect)

Pachctl inspect branch)

Pachctl inspect cluster)

Pachctl inspect commit)

Pachctl inspect datum)

Pachctl inspect file)

Pachctl inspect job)

Pachctl inspect pipeline)

Pachctl inspect project)

Pachctl inspect repo)

Pachctl inspect secret)

Pachctl inspect transaction)

Pachctl kube events)

Pachctl license)

Pachctl license activate)

Pachctl license add cluster)

Pachctl license delete all)

Pachctl license delete cluster)

Pachctl license get state)

Pachctl license list clusters)

Pachctl license update cluster)

Pachctl list branch)

Pachctl list commit)

Pachctl list datum)

Pachctl list file)

Pachctl list job)

Pachctl list pipeline)

Pachctl list project)

Pachctl list repo)

Pachctl list secret)

Pachctl list transaction)

Pachctl port forward)

Pachctl put file)

Pachctl restart)

Pachctl restart datum)

Pachctl resume)

Pachctl resume transaction)

Pachctl run cron)

Pachctl run pfs load test)

Pachctl run pps load test)

Pachctl squash)

Pachctl squash commit)

Pachctl start commit)

Pachctl start pipeline)

Pachctl start transaction)

Pachctl stop job)

Pachctl stop pipeline)

Pachctl stop transaction)

Pachctl subscribe)

Pachctl subscribe commit)

Pachctl unmount)

Pachctl update)

Pachctl update pipeline)

Pachctl update project)

Pachctl update repo)

Pachctl version)

Pachctl wait commit)

Pachctl wait job)

Configuration Specification

Helm Chart Values (HCVs)

Deploy Target HCVs

Enterprise Server HCVs

Kube Event Tail HCVs

PostgreSQL Subchart HCVs

CloudSQL Auth Proxy HCVs

OpenID Connect HCVs

Test Connection HCVs

Language Clients

Pipeline Scaling Limits (CE)

Supported Releases & Features

Troubleshooting Guides

General Troubleshooting

Troubleshooting Deployments

Troubleshooting Pipelines

Coding Conventions

Contributor Setup

Developing on Windows with VSCode

Documentation Style Guide

Get Started

New to using our platform? Start Here.

What is Pachyderm? #

Pachyderm is a data-centric pipeline and data versioning application written in go that runs on top of a Kubernetes cluster.

Local vs Cloud Installation #

Local	Cloud
Used for learning & testing.	Used in production environments.
Allocates your local machine’s resources to spin up a K8s cluster.	Uses a cloud provider (AWS, Azure, GCP) to to spin up K8s clusters.
Uses Docker Desktop, Minikube, or Kind.	Uses EKS, GKE, or AKS
Free.	Metered by cloud provider.