Archived Pachyderm Docs
🔍
🌙
2.8.x
Get Started
Beginner Tutorial
First-Time Setup
Connect to Existing Instance
Language Clients
Learn
Key Features
Target Audience
Basic Concepts
Intro to Data Versioning
Intro to Pipelines
Intro to Console
Set Cluster Defaults
Set Project Defaults
View Dashboard
View Project
Repo Actions
Create Repo
Upload Files
View Input Files
View Output Files
Delete Repo
Pipeline Actions
Create Pipelines
Duplicate Pipelines
Pause Pipelines
Rerun Pipelines
Update Pipelines
View Pipeline Details
List & Inspect Pipelines
Delete Pipelines
List & Inspect Jobs
Developer Workflow
CI/CD Integration
Create a Machine Learning Workflow
The Push Images Flag
Working with Pipelines
Diagrams
High-Level Architecture Diagram
Glossary
Ancestry Syntax
Branch
Commit
Commit Set
Cron
DAG
Data Parallelism
Datum
Deferred Processing
Distributed Computing
File
Glob Pattern
Global Identifier
History
Input Repository
Job
NLP
Output Repository
Pachyderm Worker
Pipeline
Pipeline Inputs
Pipeline Specification
Project
Provenance
Task Parallelism
User Code
Set Up
Cloud Deploy
AWS Deployment
Azure Deployment
GCP Deployment
Console Setup
Set Up AWS Secret Manager
Local Deploy
Docker Desktop
Minikube
On-Prem Deploy
Pachctl
Pachctl Auto-completion
Unified Deployment
Authentication & IdP Connectors
MockIDP
Auth0
Okta
Authorization (RBAC)
Access Control (RBAC) Roles & Permissions
Manage RBAC via Console
Add Roles to User via PachCTL
Add Roles to Group via PachCTL
Connection
Environment Variables
Kubernetes RBAC
Import a Kubernetes Context
Log Aggregation (Loki)
Non-Default Namespaces
Enterprise Edition
Activate Enterprise via Helm
Activate Enterprise via PachCTL
Features Overview
Enterprise Server (ES)
Activate ES for Multi-Cluster
Activate ES for Single-Cluster
Register a Cluster via Helm
Register a Cluster via PachCTL
Server Management
Server Setup
S3 Gateway API
TLS (SSL, HTTPS)
Tracing (Jaeger)
Global Config
Manage
Helm Chart Values (HCVs)
Deploy Target HCVs
Global HCVs
Console HCVs
Enterprise Server HCVs
ETCD HCVs
Ingress HCVs
Loki HCVs
PachD HCVs
PachW HCVs
Kube Event Tail HCVs
PGBouncer HCVs
PostgreSQL Subchart HCVs
CloudSQL Auth Proxy HCVs
OpenID Connect HCVs
Test Connection HCVs
Proxy HCVs
S3 Gateway
AWS CLI
Boto3
Credentials
MinIO
Unsupported Operations
Backup & Restore
Cluster Backup
Enterprise Server Backup
Upgrade
PachCTL Shell
Check IdP User
Supported Releases & Features
PostgresSQL Fine-Tuning
Cluster Access
Deactivate Authorization
GPUs
Log In via IdP
Revoke User Access
Scaling Limits (CE)
Secrets
Sidecar S3 Gateway
Storage Optimization
Usage Metrics
Monitor with Prometheus
Metrics
Uninstall
Prepare Data
Datum Batching
Defer Processing via Staging Branch
Ingest Data
Mount Volumes
Skip Failed Datums
Time-Windowed Data
Transactions
Build Pipelines & DAGs
Pipeline Specification (PPS)
Autoscaling PPS
Datum Set Spec PPS
Datum Timeout PPS
Datum Tries PPS
Description PPS
Egress PPS
Input Cron PPS
Input Cross PPS
Input Group PPS
Input Join PPS
Input PFS PPS
Input Union PPS
Job Timeout PPS
Metadata PPS
Output Branch PPS
Parallelism Spec PPS
Pod Patch PPS
Pod Spec PPS
Reprocess Spec PPS
Resource Limits PPS
Resource Requests PPS
s3 Out PPS
Scheduling Spec PPS
Service PPS
Sidecar Resource Limits PPS
Sidecar Resource Requests PPS
Spec Commit PPS
Spout PPS
Tolerations PPS
Transform PPS
Full Pipeline Specification
Pipeline Ops
Create a Pipeline
Delete a Pipeline
Draw a Pipeline
Inspect a Pipeline
Jsonnet Pipeline Specifications
Update a Pipeline
View Pipeline Jobs & Runtimes
Project Ops
Create a Project
Set a Project as Current
Add a Project Resource
Grant Project Access
Delete a Project
Branch Ops
Copy Files
Process Specific Commits
Set Branch Triggers
Set Output Branch
Datum Ops
Get Metadata
Inspect Datum
Provenance Ops
List Global Commits & Jobs
List Global ID Sub Commits
Track Downstream
Delete Branch Head
Squash Non-Head Commits
Delete File From History
Tutorials
Standard ML Pipeline
AutoML Pipeline
Multi-Pipeline DAG
Data Parallelism Pipeline
Task Parallelism Pipeline
Docker Image + User Code
Export Data
Egress To An SQL Database
Export via Egress
Export via PachCTL
Mount a Repo Locally
S3 Gateway Operations
Create S3 Bucket
Delete an S3 Object
Delete Empty S3 Bucket
Get an S3 Object
List S3 Buckets
List S3 Objects
Write an S3 Object
Integrate
Determined
Google BigQuery
JupyterLab
Docker Installation Guide
Run in Determined
Local Installation Guide
User Guide
Troubleshooting
Label Studio
Superb AI
VS Code Auto-Completion
Weights and Biases
Run Commands
pachctl
pachctl auth
pachctl auth activate
pachctl auth check
pachctl auth check project
pachctl auth check repo
pachctl auth deactivate
pachctl auth get
pachctl auth get cluster
pachctl auth get enterprise
pachctl auth get project
pachctl auth get repo
pachctl auth get-config
pachctl auth get-groups
pachctl auth get-robot-token
pachctl auth login
pachctl auth logout
pachctl auth revoke
pachctl auth roles-for-permission
pachctl auth rotate-root-token
pachctl auth set
pachctl auth set cluster
pachctl auth set enterprise
pachctl auth set project
pachctl auth set repo
pachctl auth set-config
pachctl auth use-auth-token
pachctl auth whoami
pachctl buildinfo
pachctl completion
pachctl completion bash
pachctl completion zsh
pachctl config
pachctl config delete
pachctl config delete context
pachctl config get
pachctl config get active-context
pachctl config get active-enterprise-context
pachctl config get context
pachctl config get metrics
pachctl config import-kube
pachctl config list
pachctl config list context
pachctl config set
pachctl config set active-context
pachctl config set active-enterprise-context
pachctl config set context
pachctl config set metrics
pachctl config update
pachctl config update context
pachctl connect
pachctl copy
pachctl copy file
pachctl create
pachctl create branch
pachctl create defaults
pachctl create pipeline
pachctl create project
pachctl create repo
pachctl create secret
pachctl debug
pachctl debug analyze
pachctl debug binary
pachctl debug dump
pachctl debug local
pachctl debug log-level
pachctl debug profile
pachctl debug template
pachctl delete
pachctl delete all
pachctl delete branch
pachctl delete commit
pachctl delete defaults
pachctl delete file
pachctl delete job
pachctl delete pipeline
pachctl delete project
pachctl delete repo
pachctl delete secret
pachctl delete transaction
pachctl diff
pachctl diff file
pachctl draw
pachctl draw pipeline
pachctl edit
pachctl edit pipeline
pachctl enterprise
pachctl enterprise deactivate
pachctl enterprise get-state
pachctl enterprise heartbeat
pachctl enterprise pause
pachctl enterprise pause-status
pachctl enterprise register
pachctl enterprise sync-contexts
pachctl enterprise unpause
pachctl exit
pachctl find
pachctl find commit
pachctl finish
pachctl finish commit
pachctl finish transaction
pachctl fsck
pachctl get
pachctl get file
pachctl glob
pachctl glob file
pachctl idp
pachctl idp create-client
pachctl idp create-connector
pachctl idp delete-client
pachctl idp delete-connector
pachctl idp get-client
pachctl idp get-config
pachctl idp get-connector
pachctl idp list-client
pachctl idp list-connector
pachctl idp set-config
pachctl idp update-client
pachctl idp update-connector
pachctl inspect
pachctl inspect branch
pachctl inspect cluster
pachctl inspect commit
pachctl inspect datum
pachctl inspect defaults
pachctl inspect file
pachctl inspect job
pachctl inspect pipeline
pachctl inspect project
pachctl inspect repo
pachctl inspect secret
pachctl inspect transaction
pachctl kube-events
pachctl license
pachctl license activate
pachctl license add-cluster
pachctl license delete-all
pachctl license delete-cluster
pachctl license get-state
pachctl license list-clusters
pachctl license update-cluster
pachctl list
pachctl list branch
pachctl list commit
pachctl list datum
pachctl list file
pachctl list job
pachctl list pipeline
pachctl list project
pachctl list repo
pachctl list secret
pachctl list transaction
pachctl logs
pachctl loki
pachctl mount
pachctl port-forward
pachctl put
pachctl put file
pachctl rerun
pachctl rerun pipeline
pachctl restart
pachctl restart datum
pachctl resume
pachctl resume transaction
pachctl run
pachctl run cron
pachctl run pfs-load-test
pachctl run pps-load-test
pachctl shell
pachctl squash
pachctl squash commit
pachctl start
pachctl start commit
pachctl start pipeline
pachctl start transaction
pachctl stop
pachctl stop job
pachctl stop pipeline
pachctl stop transaction
pachctl subscribe
pachctl subscribe commit
pachctl unmount
pachctl update
pachctl update defaults
pachctl update pipeline
pachctl update project
pachctl update repo
pachctl validate
pachctl validate pipeline
pachctl version
pachctl wait
pachctl wait commit
pachctl wait job
Debug
Common Issues
Debug Pipelines
Troubleshooting Deployments
View Audit Logs
View Kubernetes Logs
Pachyderm SDK
Client Initialization (Start Here)
Method Mapping
First Project
Examples
Breast Cancer Detection
Distributed Image Processing
Spout Pipelines
Reference Docs
Contribute
Coding Conventions
Contributor Setup
Developing on Windows with VSCode
Documentation Style Guide
2.7.x
Get Started
Beginner Tutorial
First-Time Setup
Connect to Existing Instance
Language Clients
Learn
Key Features
Target Audience
Basic Concepts
Intro to Data Versioning
Intro to Pipelines
Intro to Console
View Dashboard
View Project
View List
View Pipelines
View Jobs
View Outputs
View Inputs
Developer Workflow
CI/CD Integration
Create a Machine Learning Workflow
The Push Images Flag
Working with Pipelines
Diagrams
High-Level Architecture Diagram
Glossary
Ancestry Syntax
Branch
Commit
Commit Set
Cron
DAG
Data Parallelism
Datum
Deferred Processing
Distributed Computing
File
Glob Pattern
Global Identifier
History
Input Repository
Job
NLP
Output Repository
Pachyderm Worker
Pipeline
Pipeline Inputs
Pipeline Specification
Project
Provenance
Task Parallelism
User Code
Set Up
Cloud Deploy
AWS Deployment
Azure Deployment
GCP Deployment
Console Setup
Set Up AWS Secret Manager
Local Deploy
Docker Desktop
Minikube
On-Prem Deploy
Pachctl
Pachctl Auto-completion
Authentication & IdP Connectors
MockIDP
Auth0
Okta
Authorization (RBAC)
Access Control (RBAC) Roles & Permissions
Manage RBAC via Console
Add Roles to User via PachCTL
Add Roles to Group via PachCTL
Connection
Environment Variables
Kubernetes RBAC
Import a Kubernetes Context
Log Aggregation (Loki)
Non-Default Namespaces
Enterprise Edition
Activate Enterprise via Helm
Activate Enterprise via PachCTL
Features Overview
Enterprise Server (ES)
Activate ES for Multi-Cluster
Activate ES for Single-Cluster
Register a Cluster via Helm
Register a Cluster via PachCTL
Server Management
Server Setup
S3 Gateway API
TLS (SSL, HTTPS)
Tracing (Jaeger)
Manage
Helm Chart Values (HCVs)
Deploy Target HCVs
Global HCVs
Console HCVs
Enterprise Server HCVs
ETCD HCVs
Ingress HCVs
Loki HCVs
PachD HCVs
PachW HCVs
Kube Event Tail HCVs
PGBouncer HCVs
PostgreSQL Subchart HCVs
CloudSQL Auth Proxy HCVs
OpenID Connect HCVs
Test Connection HCVs
Proxy HCVs
S3 Gateway
AWS CLI
Boto3
Credentials
MinIO
Unsupported Operations
Backup & Restore
Cluster Backup
Enterprise Server Backup
Upgrade
PachCTL Shell
Check IdP User
Supported Releases & Features
PostgresSQL Fine-Tuning
Cluster Access
Deactivate Authorization
GPUs
Log In via IdP
Revoke User Access
Scaling Limits (CE)
Secrets
Sidecar S3 Gateway
Storage Optimization
Usage Metrics
Monitor with Prometheus
Metrics
Uninstall
Prepare Data
Datum Batching
Defer Processing via Staging Branch
Ingest Data
Mount Volumes
Skip Failed Datums
SQL Ingest
Time-Windowed Data
Transactions
Build Pipelines & DAGs
Pipeline Specification (PPS)
Autoscaling PPS
Datum Set Spec PPS
Datum Timeout PPS
Datum Tries PPS
Description PPS
Egress PPS
Input Cron PPS
Input Cross PPS
Input Group PPS
Input Join PPS
Input PFS PPS
Input Union PPS
Job Timeout PPS
Metadata PPS
Output Branch PPS
Parallelism Spec PPS
Pod Patch PPS
Pod Spec PPS
Reprocess Spec PPS
Resource Limits PPS
Resource Requests PPS
s3 Out PPS
Scheduling Spec PPS
Service PPS
Sidecar Resource Limits PPS
Sidecar Resource Requests PPS
Spec Commit PPS
Spout PPS
Transform PPS
Full Pipeline Specification
Pipeline Ops
Create a Pipeline
Delete a Pipeline
Draw a Pipeline
Inspect a Pipeline
Jsonnet Pipeline Specifications
Update a Pipeline
View Pipeline Jobs & Runtimes
Project Ops
Create a Project
Set a Project as Current
Add a Project Resource
Grant Project Access
Delete a Project
Branch Ops
Copy Files
Process Specific Commits
Set Branch Triggers
Set Output Branch
Datum Ops
Get Metadata
Inspect Datum
Provenance Ops
List Global Commits & Jobs
List Global ID Sub Commits
Track Downstream
Delete Branch Head
Squash Non-Head Commits
Delete File From History
Tutorials
Standard ML Pipeline
AutoML Pipeline
Multi-Pipeline DAG
Data Parallelism Pipeline
Task Parallelism Pipeline
Docker Image + User Code
Export Data
Egress To An SQL Database
Export via Egress
Export via PachCTL
Mount a Repo Locally
S3 Gateway Operations
Create S3 Bucket
Delete an S3 Object
Delete Empty S3 Bucket
Get an S3 Object
List S3 Buckets
List S3 Objects
Write an S3 Object
Integrate
Determined
Google BigQuery
JupyterLab
Docker Installation Guide
Run in Determined
Local Installation Guide
User Guide
Troubleshooting
Label Studio
Superb AI
Weights and Biases
Run Commands
pachctl
pachctl auth
pachctl auth activate
pachctl auth check
pachctl auth check project
pachctl auth check repo
pachctl auth deactivate
pachctl auth get
pachctl auth get cluster
pachctl auth get enterprise
pachctl auth get project
pachctl auth get repo
pachctl auth get-config
pachctl auth get-groups
pachctl auth get-robot-token
pachctl auth login
pachctl auth logout
pachctl auth revoke
pachctl auth roles-for-permission
pachctl auth rotate-root-token
pachctl auth set
pachctl auth set cluster
pachctl auth set enterprise
pachctl auth set project
pachctl auth set repo
pachctl auth set-config
pachctl auth use-auth-token
pachctl auth whoami
pachctl buildinfo
pachctl completion
pachctl completion bash
pachctl completion zsh
pachctl config
pachctl config delete
pachctl config delete context
pachctl config get
pachctl config get active-context
pachctl config get active-enterprise-context
pachctl config get context
pachctl config get metrics
pachctl config import-kube
pachctl config list
pachctl config list context
pachctl config set
pachctl config set active-context
pachctl config set active-enterprise-context
pachctl config set context
pachctl config set metrics
pachctl config update
pachctl config update context
pachctl connect
pachctl copy
pachctl copy file
pachctl create
pachctl create branch
pachctl create pipeline
pachctl create project
pachctl create repo
pachctl create secret
pachctl debug
pachctl debug analyze
pachctl debug binary
pachctl debug dump
pachctl debug log-level
pachctl debug profile
pachctl debug template
pachctl delete
pachctl delete all
pachctl delete branch
pachctl delete commit
pachctl delete file
pachctl delete job
pachctl delete pipeline
pachctl delete project
pachctl delete repo
pachctl delete secret
pachctl delete transaction
pachctl diff
pachctl diff file
pachctl draw
pachctl draw pipeline
pachctl edit
pachctl edit pipeline
pachctl enterprise
pachctl enterprise deactivate
pachctl enterprise get-state
pachctl enterprise heartbeat
pachctl enterprise pause
pachctl enterprise pause-status
pachctl enterprise register
pachctl enterprise sync-contexts
pachctl enterprise unpause
pachctl exit
pachctl find
pachctl find commit
pachctl finish
pachctl finish commit
pachctl finish transaction
pachctl fsck
pachctl get
pachctl get file
pachctl glob
pachctl glob file
pachctl idp
pachctl idp create-client
pachctl idp create-connector
pachctl idp delete-client
pachctl idp delete-connector
pachctl idp get-client
pachctl idp get-config
pachctl idp get-connector
pachctl idp list-client
pachctl idp list-connector
pachctl idp set-config
pachctl idp update-client
pachctl idp update-connector
pachctl inspect
pachctl inspect branch
pachctl inspect cluster
pachctl inspect commit
pachctl inspect datum
pachctl inspect file
pachctl inspect job
pachctl inspect pipeline
pachctl inspect project
pachctl inspect repo
pachctl inspect secret
pachctl inspect transaction
pachctl kube-events
pachctl license
pachctl license activate
pachctl license add-cluster
pachctl license delete-all
pachctl license delete-cluster
pachctl license get-state
pachctl license list-clusters
pachctl license update-cluster
pachctl list
pachctl list branch
pachctl list commit
pachctl list datum
pachctl list file
pachctl list job
pachctl list pipeline
pachctl list project
pachctl list repo
pachctl list secret
pachctl list transaction
pachctl logs
pachctl loki
pachctl mount
pachctl port-forward
pachctl put
pachctl put file
pachctl restart
pachctl restart datum
pachctl resume
pachctl resume transaction
pachctl run
pachctl run cron
pachctl run pfs-load-test
pachctl run pps-load-test
pachctl shell
pachctl squash
pachctl squash commit
pachctl start
pachctl start commit
pachctl start pipeline
pachctl start transaction
pachctl stop
pachctl stop job
pachctl stop pipeline
pachctl stop transaction
pachctl subscribe
pachctl subscribe commit
pachctl unmount
pachctl update
pachctl update pipeline
pachctl update project
pachctl update repo
pachctl validate
pachctl validate pipeline
pachctl version
pachctl wait
pachctl wait commit
pachctl wait job
Debug
Common Issues
Debug Pipelines
Troubleshooting Deployments
View Audit Logs
View Kubernetes Logs
Pachyderm SDK
Client Initialization (Start Here)
Method Mapping
First Project
Examples
Breast Cancer Detection
Distributed Image Processing
Spout Pipelines
Reference Docs
Contribute
Coding Conventions
Contributor Setup
Developing on Windows with VSCode
Documentation Style Guide
Shared
2.6.x
Get Started
Beginner Tutorial
First-Time Setup
Connect to Existing Instance
Language Clients
Learn
Key Features
Target Audience
Basic Concepts
Intro to Data Versioning
Intro to Pipelines
Intro to Console
View Project
View List
View Pipelines
View Jobs
View Outputs
View Inputs
Developer Workflow
CI/CD Integration
Create a Machine Learning Workflow
The Push Images Flag
Working with Pipelines
Diagrams
High-Level Architecture Diagram
Glossary
Ancestry Syntax
Branch
Commit
Commit Set
DAG
Data Parallelism
Datum
Deferred Processing
Distributed Computing
File
Glob Pattern
Global Identifier
History
Input Repository
Job
NLP
Output Repository
Pachyderm Worker
Pipeline
Pipeline Inputs
Pipeline Specification
Provenance
Task Parallelism
User Code
Set Up
Cloud Deploy
AWS Deployment
Azure Deployment
GCP Deployment
Console Setup
Set Up AWS Secret Manager
Local Deploy
Docker Desktop
Minikube
On-Prem Deploy
Pachctl
Pachctl Auto-completion
Authentication & IdP Connectors
Auth0
Okta
Authorization
Add Roles to User
Add Roles to Group
IAM
Connection
Environment Variables
Kubernetes RBAC
Import a Kubernetes Context
Log Aggregation (Loki)
Non-Default Namespaces
Enterprise Edition
Activate Enterprise via Helm
Activate Enterprise via PachCTL
Features Overview
Enterprise Server (ES)
Activate ES for Multi-Cluster
Activate ES for Single-Cluster
Register a Cluster via Helm
Register a Cluster via PachCTL
Server Management
Server Setup
S3 Gateway API
TLS (SSL, HTTPS)
Tracing (Jaeger)
Manage
Helm Chart Values (HCVs)
Deploy Target HCVs
Global HCVs
Console HCVs
Enterprise Server HCVs
ETCD HCVs
Ingress HCVs
Loki HCVs
PachD HCVs
PachW HCVs
Kube Event Tail HCVs
PGBouncer HCVs
PostgreSQL Subchart HCVs
CloudSQL Auth Proxy HCVs
OpenID Connect HCVs
Test Connection HCVs
Proxy HCVs
S3 Gateway
AWS CLI
Boto3
Credentials
MinIO
Unsupported Operations
Backup & Restore
Cluster Backup
Enterprise Server Backup
Upgrade
PachCTL Shell
Check IdP User
Supported Releases & Features
Cluster Access
Deactivate Authorization
GPUs
Log In via IdP
Revoke User Access
Scaling Limits (CE)
Secrets
Sidecar S3 Gateway
Storage Optimization
Usage Metrics
Monitor with Prometheus
Metrics
Uninstall
Prepare Data
Datum Batching
Defer Processing via Staging Branch
Ingest Data
Mount Volumes
Skip Failed Datums
SQL Ingest
Time-Windowed Data
Transactions
Build Pipelines & DAGs
Pipeline Specification (PPS)
Autoscaling PPS
Datum Set Spec PPS
Datum Timeout PPS
Datum Tries PPS
Description PPS
Egress PPS
Input Cron PPS
Input Cross PPS
Input Group PPS
Input Join PPS
Input PFS PPS
Input Union PPS
Job Timeout PPS
Metadata PPS
Output Branch PPS
Parallelism Spec PPS
Pod Patch PPS
Pod Spec PPS
Reprocess Spec PPS
Resource Limits PPS
Resource Requests PPS
s3 Out PPS
Scheduling Spec PPS
Service PPS
Sidecar Resource Limits PPS
Sidecar Resource Requests PPS
Spec Commit PPS
Spout PPS
Transform PPS
Full Pipeline Specification
Pipeline Ops
Create a Pipeline
Delete a Pipeline
Jsonnet Pipeline Specifications
Update a Pipeline
Project Ops
Create a Project
Set a Project as Current
Add a Project Resource
Grant Project Access
Delete a Project
Branch Ops
Copy Files
Process Specific Commits
Set Branch Triggers
Set Output Branch
Datum Ops
Get Metadata
Inspect Datum
Provenance Ops
List Global Commits & Jobs
List Global ID Sub Commits
Track Downstream
Delete Branch Head
Squash Non-Head Commits
Delete File From History
Tutorials
Standard ML Pipeline
AutoML Pipeline
Multi-Pipeline DAG
Data Parallelism Pipeline
Task Parallelism Pipeline
Docker Image + User Code
Export Data
Egress To An SQL Database
Export via Egress
Export via PachCTL
Mount a Repo Locally
S3 Gateway Operations
Create S3 Bucket
Delete an S3 Object
Delete Empty S3 Bucket
Get an S3 Object
List S3 Buckets
List S3 Objects
Write an S3 Object
Integrate
Google BigQuery
JupyterLab
Docker Installation Guide
Local Installation Guide
User Guide
Troubleshooting
Label Studio
Superb AI
Weights and Biases
Run Commands
Pachctl)
Pachctl auth)
Pachctl auth activate)
Pachctl auth check)
Pachctl auth check project)
Pachctl auth check repo)
Pachctl auth deactivate)
Pachctl auth get config)
Pachctl auth get groups)
Pachctl auth get robot token)
Pachctl auth get)
Pachctl auth get cluster)
Pachctl auth get enterprise)
Pachctl auth get project)
Pachctl auth get repo)
Pachctl auth login)
Pachctl auth logout)
Pachctl auth revoke)
Pachctl auth roles for permission)
Pachctl auth rotate root token)
Pachctl auth set config)
Pachctl auth set)
Pachctl auth set cluster)
Pachctl auth set enterprise)
Pachctl auth set project)
Pachctl auth set repo)
Pachctl auth use auth token)
Pachctl auth whoami)
Pachctl buildinfo)
Pachctl completion)
Pachctl completion bash)
Pachctl completion zsh)
Pachctl config)
Pachctl config delete)
Pachctl config delete context)
Pachctl config get)
Pachctl config get active context)
Pachctl config get active enterprise context)
Pachctl config get context)
Pachctl config get metrics)
Pachctl config import kube)
Pachctl config list)
Pachctl config list context)
Pachctl config set)
Pachctl config set active context)
Pachctl config set active enterprise context)
Pachctl config set context)
Pachctl config set metrics)
Pachctl config update)
Pachctl config update context)
Pachctl connect)
Pachctl copy)
Pachctl copy file)
Pachctl create)
Pachctl create branch)
Pachctl create pipeline)
Pachctl create project)
Pachctl create repo)
Pachctl create secret)
Pachctl debug)
Pachctl debug analyze)
Pachctl debug binary)
Pachctl debug dump)
Pachctl debug log level)
Pachctl debug profile)
Pachctl delete)
Pachctl delete all)
Pachctl delete branch)
Pachctl delete commit)
Pachctl delete file)
Pachctl delete job)
Pachctl delete pipeline)
Pachctl delete project)
Pachctl delete repo)
Pachctl delete secret)
Pachctl delete transaction)
Pachctl diff)
Pachctl diff file)
Pachctl draw)
Pachctl draw pipeline)
Pachctl edit)
Pachctl edit pipeline)
Pachctl enterprise)
Pachctl enterprise deactivate)
Pachctl enterprise get state)
Pachctl enterprise heartbeat)
Pachctl enterprise pause status)
Pachctl enterprise pause)
Pachctl enterprise register)
Pachctl enterprise sync contexts)
Pachctl enterprise unpause)
Pachctl exit)
Pachctl find)
Pachctl find commit)
Pachctl finish)
Pachctl finish commit)
Pachctl finish transaction)
Pachctl fsck)
Pachctl get)
Pachctl get file)
Pachctl glob)
Pachctl glob file)
Pachctl idp)
Pachctl idp create client)
Pachctl idp create connector)
Pachctl idp delete client)
Pachctl idp delete connector)
Pachctl idp get client)
Pachctl idp get config)
Pachctl idp get connector)
Pachctl idp list client)
Pachctl idp list connector)
Pachctl idp set config)
Pachctl idp update client)
Pachctl idp update connector)
Pachctl inspect)
Pachctl inspect branch)
Pachctl inspect cluster)
Pachctl inspect commit)
Pachctl inspect datum)
Pachctl inspect file)
Pachctl inspect job)
Pachctl inspect pipeline)
Pachctl inspect project)
Pachctl inspect repo)
Pachctl inspect secret)
Pachctl inspect transaction)
Pachctl kube events)
Pachctl license)
Pachctl license activate)
Pachctl license add cluster)
Pachctl license delete all)
Pachctl license delete cluster)
Pachctl license get state)
Pachctl license list clusters)
Pachctl license update cluster)
Pachctl list)
Pachctl list branch)
Pachctl list commit)
Pachctl list datum)
Pachctl list file)
Pachctl list job)
Pachctl list pipeline)
Pachctl list project)
Pachctl list repo)
Pachctl list secret)
Pachctl list transaction)
Pachctl logs)
Pachctl loki)
Pachctl mount)
Pachctl port forward)
Pachctl put)
Pachctl put file)
Pachctl restart)
Pachctl restart datum)
Pachctl resume)
Pachctl resume transaction)
Pachctl run)
Pachctl run cron)
Pachctl run pfs load test)
Pachctl run pps load test)
Pachctl shell)
Pachctl squash)
Pachctl squash commit)
Pachctl start)
Pachctl start commit)
Pachctl start pipeline)
Pachctl start transaction)
Pachctl stop)
Pachctl stop job)
Pachctl stop pipeline)
Pachctl stop transaction)
Pachctl subscribe)
Pachctl subscribe commit)
Pachctl unmount)
Pachctl update)
Pachctl update pipeline)
Pachctl update project)
Pachctl update repo)
Pachctl version)
Pachctl wait)
Pachctl wait commit)
Pachctl wait job)
Debug
Common Issues
Debug Pipelines
Troubleshooting Deployments
View Audit Logs
View Kubernetes Logs
SDKs
Contribute
Coding Conventions
Contributor Setup
Developing on Windows with VSCode
Documentation Style Guide
Shared
2.4.x
Get Started
Local Getting Started Guides
Docker Desktop
Minikube
Cloud Getting Started Guides
AWS Deployment
Azure Deployment
GCP Deployment
Install PachCTL Auto-completion
Beginner Tutorial
Concepts
Advanced Concepts
Deferred Processing
Distributed Computing
Global Identifier
Pipeline Concepts
Datum
Cross & Union Inputs
Datum Processing
Datum Processing States
Glob Pattern
Group Input
Join Input
Metadata
Job
Pipeline
Cron Pipeline
Service
Spout
Versioned Data Concepts
Branch
Commit
File
History
Provenance
Repository
How Tos
Advanced Data Operations
Analyze Time-Windowed Data
Manage Secrets
Mount a Volume in a Pipeline
Skip Failed Datums
Transactions
Basic Data Operations
Export Data
Egress To An SQL Database
Export via Egress
Export via PachCTL
Mount a Repo Locally
Ingest Data
Manage Commits & Delete Data
SQL Ingest
Developer Workflow
CI/CD Integration
Create a Machine Learning Workflow
The Push Images Flag
Working with Pipelines
JupyterLab Extension
Docker Installation Guide
Extension Walkthrough
Local Installation Guide
User Guide
Troubleshooting
Pipeline Operations
Create a Pipeline
Delete a Pipeline
Jsonnet Pipeline Specifications
Update a Pipeline
Deploy & Manage
Deploy
Azure
Configure Environment Variables
Configure Tracing with Jaeger
Connect to a Cluster
Console
Cloud Deployment
Local Deployment
Deploy on AWS
Deploy via Proxy (One Port)
Deploy with TLS (SSL, HTTPS)
Enable Logs Aggregation With Loki
Google Cloud Platform
Helm Deployment
Import a Kubernetes Context
Infrastructure Recommendations
Setup Ingress with Traefik
Monitor with Prometheus
Job Metrics
Pachd Metrics
Non-Default Namespaces
On Premises
Production Recommendations
Quickstart
RBAC
Set Up AWS Secret Manager
Manage
Backup & Restore Your Cluster
Disable Usage Metrics
Global S3 Gateway
S3 Client Configuration
S3 Gateway Supported Operations
Unsupported operations
Manage Cluster Access
Sidecar S3 Gateway
Storage Use Optimization
Uninstall
Upgrade
Use GPUs
Using the PachCTL Shell
Reference
PachCTL
pachctl auth
pachctl auth activate
pachctl auth check
pachctl auth check repo
pachctl auth deactivate
pachctl auth get
pachctl auth get cluster
pachctl auth get enterprise
pachctl auth get repo
pachctl auth get-config
pachctl auth get-groups
pachctl auth get-robot-token
pachctl auth login
pachctl auth logout
pachctl auth revoke
pachctl auth roles-for-permission
pachctl auth rotate-root-token
pachctl auth set
pachctl auth set cluster
pachctl auth set enterprise
pachctl auth set repo
pachctl auth set-config
pachctl auth use-auth-token
pachctl auth whoami
pachctl buildinfo
pachctl completion
pachctl completion bash
pachctl completion zsh
pachctl config
pachctl config delete
pachctl config delete context
pachctl config get
pachctl config get active-context
pachctl config get active-enterprise-context
pachctl config get context
pachctl config get metrics
pachctl config import-kube
pachctl config list
pachctl config list context
pachctl config set
pachctl config set active-context
pachctl config set active-enterprise-context
pachctl config set context
pachctl config set metrics
pachctl config update
pachctl config update context
pachctl copy
pachctl copy file
pachctl create
pachctl create branch
pachctl create pipeline
pachctl create repo
pachctl create secret
pachctl debug
pachctl debug analyze
pachctl debug binary
pachctl debug dump
pachctl debug profile
pachctl delete
pachctl delete all
pachctl delete branch
pachctl delete commit
pachctl delete file
pachctl delete job
pachctl delete pipeline
pachctl delete repo
pachctl delete secret
pachctl delete transaction
pachctl diff
pachctl diff file
pachctl draw
pachctl edit
pachctl edit pipeline
pachctl enterprise
pachctl enterprise deactivate
pachctl enterprise get-state
pachctl enterprise heartbeat
pachctl enterprise pause
pachctl enterprise pause-status
pachctl enterprise register
pachctl enterprise sync-contexts
pachctl enterprise unpause
pachctl exit
pachctl finish
pachctl finish commit
pachctl finish transaction
pachctl fsck
pachctl get
pachctl get file
pachctl glob
pachctl glob file
pachctl idp
pachctl idp create-client
pachctl idp create-connector
pachctl idp delete-client
pachctl idp delete-connector
pachctl idp get-client
pachctl idp get-config
pachctl idp get-connector
pachctl idp list-client
pachctl idp list-connector
pachctl idp set-config
pachctl idp update-client
pachctl idp update-connector
pachctl inspect
pachctl inspect branch
pachctl inspect cluster
pachctl inspect commit
pachctl inspect datum
pachctl inspect file
pachctl inspect job
pachctl inspect pipeline
pachctl inspect repo
pachctl inspect secret
pachctl inspect transaction
pachctl license
pachctl license activate
pachctl license add-cluster
pachctl license delete-all
pachctl license delete-cluster
pachctl license get-state
pachctl license list-clusters
pachctl license update-cluster
pachctl list
pachctl list branch
pachctl list commit
pachctl list datum
pachctl list file
pachctl list job
pachctl list pipeline
pachctl list repo
pachctl list secret
pachctl list transaction
pachctl logs
pachctl mount
pachctl port-forward
pachctl put
pachctl put file
pachctl restart
pachctl restart datum
pachctl resume
pachctl resume transaction
pachctl run
pachctl run cron
pachctl run pfs-load-test
pachctl run pps-load-test
pachctl shell
pachctl squash
pachctl squash commit
pachctl start
pachctl start commit
pachctl start pipeline
pachctl start transaction
pachctl stop
pachctl stop job
pachctl stop pipeline
pachctl stop transaction
pachctl subscribe
pachctl subscribe commit
pachctl unmount
pachctl update
pachctl update pipeline
pachctl update repo
pachctl version
pachctl wait
pachctl wait commit
pachctl wait job
Configuration Specification
Helm Chart Values
Language Clients
Pipeline Scaling Limits (CE)
Pipeline Specification
S3 Gateway API
Supported Releases & Features
Enterprise Edition
Authentication & Authorization
Authentication
Authentication Flows
Connectors
Okta
Identity Providers
Authorization
Permissions by Role
Role Binding
Enterprise Server
Server Management
Server Setup
Deploy
Features Overview
Troubleshooting Guides
General Troubleshooting
Troubleshooting Deployments
Troubleshooting Pipelines
Contributing
Coding Conventions
Contributor Setup
Developing on Windows with VSCode
Documentation Style Guide
2.5.x
Overview
Key Features
Target Audience
Basic Concepts
Intro to Data Versioning
Intro to Pipelines
Get Started
Local Getting Started Guides
Docker Desktop
Minikube
Cloud Getting Started Guides
AWS Deployment
Azure Deployment
GCP Deployment
Install PachCTL Auto-completion
Beginner Tutorial
Concepts
Advanced Concepts
Deferred Processing
Distributed Computing
Global Identifier
Pipeline Concepts
Datum
Cross & Union Inputs
Datum Processing
Datum Processing States
Glob Pattern
Group Input
Join Input
Metadata
Job
Pipeline
Cron Pipeline
Service
Spout
Versioned Data Concepts
Branch
Commit
File
History
Provenance
Repository
How Tos
Advanced Data Operations
Analyze Time-Windowed Data
Manage Secrets
Mount a Volume in a Pipeline
Skip Failed Datums
Transactions
Basic Data Operations
Export Data
Egress To An SQL Database
Export via Egress
Export via PachCTL
Mount a Repo Locally
Ingest Data
Manage Commits & Delete Data
SQL Ingest
Console Guide
View Project
View List
View Pipelines
View Jobs
View Outputs
View Inputs
Debugging Operations
View Audit Logs
View Kubernetes Logs
Developer Workflow
CI/CD Integration
Create a Machine Learning Workflow
The Push Images Flag
Working with Pipelines
JupyterLab Mount Extension
Docker Installation Guide
Local Installation Guide
User Guide
Troubleshooting
Pipeline Operations
Create a Pipeline
Delete a Pipeline
Jsonnet Pipeline Specifications
Update a Pipeline
Project Operations
Create a Project
Set a Project as Current
Add a Project Resource
Grant Project Access
Delete a Project
S3 Gateway Operations
Create S3 Bucket
Delete an S3 Object
Delete Empty S3 Bucket
Get an S3 Object
List S3 Buckets
List S3 Objects
Write an S3 Object
Deploy & Manage
Deploy
AWS Deployment
Azure
Configure Environment Variables
Configure Tracing with Jaeger
Console
Cloud Deployment
Local Deployment
Deploy with TSL (SSL, HTTPS)
Enable Logs Aggregation With Loki
Google Cloud Platform
Helm Deployment
Import a Kubernetes Context
Infrastructure Recommendations
Setup Ingress with Traefik
Monitor with Prometheus
Job Metrics
Pachd Metrics
Non-Default Namespaces
On Premises
Production Recommendations
Quickstart
RBAC
Set Up AWS Secret Manager
Upgrade to Embedded Proxy
Manage
Backup & Restore Your Cluster
Disable Usage Metrics
Manage Cluster Access
S3 Gateway
AWS CLI
Boto3
Credentials
MinIO
Unsupported Operations
Sidecar S3 Gateway
Storage Use Optimization
Uninstall
Upgrade
Use GPUs
Using the PachCTL Shell
Reference
PachCTL
Pachctl auth)
Pachctl auth activate)
Pachctl auth check)
Pachctl auth check project)
Pachctl auth check repo)
Pachctl auth deactivate)
Pachctl auth get config)
Pachctl auth get groups)
Pachctl auth get robot token)
Pachctl auth get)
Pachctl auth get cluster)
Pachctl auth get enterprise)
Pachctl auth get project)
Pachctl auth get repo)
Pachctl auth login)
Pachctl auth logout)
Pachctl auth revoke)
Pachctl auth roles for permission)
Pachctl auth rotate root token)
Pachctl auth set config)
Pachctl auth set)
Pachctl auth set cluster)
Pachctl auth set enterprise)
Pachctl auth set project)
Pachctl auth set repo)
Pachctl auth use auth token)
Pachctl auth whoami)
Pachctl buildinfo)
Pachctl completion)
Pachctl completion bash)
Pachctl completion zsh)
Pachctl config)
Pachctl config delete)
Pachctl config delete context)
Pachctl config get)
Pachctl config get active context)
Pachctl config get active enterprise context)
Pachctl config get context)
Pachctl config get metrics)
Pachctl config import kube)
Pachctl config list)
Pachctl config list context)
Pachctl config set)
Pachctl config set active context)
Pachctl config set active enterprise context)
Pachctl config set context)
Pachctl config set metrics)
Pachctl config update)
Pachctl config update context)
Pachctl connect)
Pachctl copy)
Pachctl copy file)
Pachctl create)
Pachctl create branch)
Pachctl create pipeline)
Pachctl create project)
Pachctl create repo)
Pachctl create secret)
Pachctl debug)
Pachctl debug analyze)
Pachctl debug binary)
Pachctl debug dump)
Pachctl debug profile)
Pachctl delete)
Pachctl delete all)
Pachctl delete branch)
Pachctl delete commit)
Pachctl delete file)
Pachctl delete job)
Pachctl delete pipeline)
Pachctl delete project)
Pachctl delete repo)
Pachctl delete secret)
Pachctl delete transaction)
Pachctl diff)
Pachctl diff file)
Pachctl draw)
Pachctl draw pipeline)
Pachctl edit)
Pachctl edit pipeline)
Pachctl enterprise)
Pachctl enterprise deactivate)
Pachctl enterprise get state)
Pachctl enterprise heartbeat)
Pachctl enterprise pause status)
Pachctl enterprise pause)
Pachctl enterprise register)
Pachctl enterprise sync contexts)
Pachctl enterprise unpause)
Pachctl exit)
Pachctl finish)
Pachctl finish commit)
Pachctl finish transaction)
Pachctl fsck)
Pachctl get)
Pachctl get file)
Pachctl glob)
Pachctl glob file)
Pachctl idp)
Pachctl idp create client)
Pachctl idp create connector)
Pachctl idp delete client)
Pachctl idp delete connector)
Pachctl idp get client)
Pachctl idp get config)
Pachctl idp get connector)
Pachctl idp list client)
Pachctl idp list connector)
Pachctl idp set config)
Pachctl idp update client)
Pachctl idp update connector)
Pachctl inspect)
Pachctl inspect branch)
Pachctl inspect cluster)
Pachctl inspect commit)
Pachctl inspect datum)
Pachctl inspect file)
Pachctl inspect job)
Pachctl inspect pipeline)
Pachctl inspect project)
Pachctl inspect repo)
Pachctl inspect secret)
Pachctl inspect transaction)
Pachctl kube events)
Pachctl license)
Pachctl license activate)
Pachctl license add cluster)
Pachctl license delete all)
Pachctl license delete cluster)
Pachctl license get state)
Pachctl license list clusters)
Pachctl license update cluster)
Pachctl list)
Pachctl list branch)
Pachctl list commit)
Pachctl list datum)
Pachctl list file)
Pachctl list job)
Pachctl list pipeline)
Pachctl list project)
Pachctl list repo)
Pachctl list secret)
Pachctl list transaction)
Pachctl logs)
Pachctl mount)
Pachctl port forward)
Pachctl put)
Pachctl put file)
Pachctl restart)
Pachctl restart datum)
Pachctl resume)
Pachctl resume transaction)
Pachctl run)
Pachctl run cron)
Pachctl run pfs load test)
Pachctl run pps load test)
Pachctl shell)
Pachctl squash)
Pachctl squash commit)
Pachctl start)
Pachctl start commit)
Pachctl start pipeline)
Pachctl start transaction)
Pachctl stop)
Pachctl stop job)
Pachctl stop pipeline)
Pachctl stop transaction)
Pachctl subscribe)
Pachctl subscribe commit)
Pachctl unmount)
Pachctl update)
Pachctl update pipeline)
Pachctl update project)
Pachctl update repo)
Pachctl version)
Pachctl wait)
Pachctl wait commit)
Pachctl wait job)
Configuration Specification
Helm Chart Values (HCVs)
Deploy Target HCVs
Global HCVs
Console HCVs
Enterprise Server HCVs
ETCD HCVs
Ingress HCVs
Loki HCVs
PachD HCVs
PachW HCVs
Kube Event Tail HCVs
PGBouncer HCVs
PostgreSQL Subchart HCVs
CloudSQL Auth Proxy HCVs
OpenID Connect HCVs
Test Connection HCVs
Proxy HCVs
Language Clients
Pipeline Scaling Limits (CE)
Pipeline Specification (PPS)
Autoscaling PPS
Datum Set Spec PPS
Datum Timeout PPS
Datum Tries PPS
Description PPS
Egress PPS
Input Cron PPS
Input Cross PPS
Input Group PPS
Input Join PPS
Input PFS PPS
Input Union PPS
Job Timeout PPS
Metadata PPS
Output Branch PPS
Parallelism Spec PPS
Pod Patch PPS
Pod Spec PPS
Reprocess Spec PPS
Resource Limits PPS
Resource Requests PPS
s3 Out PPS
Scheduling Spec PPS
Service PPS
Sidecar Resource Limits PPS
Spec Commit PPS
Spout PPS
Transform PPS
S3 Gateway API
Supported Releases & Features
Enterprise Edition
Activate Enterprise via Helm
Activate Enterprise via PachCTL
Authentication & Authorization
Authentication
Activate Authorization
Set Up IdP Connectors
Auth0
Okta
Log In via IdP
Check IdP User
Revoke User Access
Deactivate Authorization
Authorization
Add Roles to User
Add Roles to Group
IAM
Enterprise Server (ES)
Activate ES for Multi-Cluster
Activate ES for Single-Cluster
Register a Cluster via Helm
Register a Cluster via PachCTL
Server Management
Server Setup
Features Overview
Troubleshooting Guides
General Troubleshooting
Troubleshooting Deployments
Troubleshooting Pipelines
Contributing
Coding Conventions
Contributor Setup
Developing on Windows with VSCode
Documentation Style Guide
Changelog
Latest
Get Started
Beginner Tutorial
First-Time Setup
Connect to Existing Instance
Language Clients
Learn
Key Features
Target Audience
Basic Concepts
Intro to Data Versioning
Intro to Pipelines
Intro to Console
Set Cluster Defaults
Set Project Defaults
View Dashboard
View Project
Repo Actions
Create Repo
Upload Files
View Input Files
View Output Files
Delete Repo
Pipeline Actions
Create Pipelines
Duplicate Pipelines
Pause/Restart Pipelines
Rerun Pipelines
Update Pipelines
View Pipeline Details
List & Inspect Pipelines
Delete Pipelines
List & Inspect Jobs
Developer Workflow
CI/CD Integration
Create a Machine Learning Workflow
The Push Images Flag
Working with Pipelines
Diagrams
High-Level Architecture Diagram
Glossary
Ancestry Syntax
Branch
Commit
Commit Set
Cron
DAG
Data Parallelism
Datum
Deferred Processing
Distributed Computing
File
Glob Pattern
Global Identifier
History
Input Repository
Job
NLP
Output Repository
Pachyderm Worker
Pipeline
Pipeline Inputs
Pipeline Specification
Project
Provenance
Task Parallelism
User Code
Set Up
Cloud Deploy
AWS Deployment
Azure Deployment
GCP Deployment
Console Setup
Set Up AWS Secret Manager
Local Deploy
Docker Desktop
Minikube
On-Prem Deploy
Pachctl
Pachctl Auto-completion
Unified Deployment
Authentication & IdP Connectors
MockIDP
Auth0
Okta
Authorization (RBAC)
Access Control (RBAC) Roles & Permissions
Manage RBAC via Console
Add Roles to User via PachCTL
Add Roles to Group via PachCTL
Connection
Environment Variables
Kubernetes RBAC
Import a Kubernetes Context
Log Aggregation (Loki)
Non-Default Namespaces
Enterprise Edition
Activate Enterprise via Helm
Activate Enterprise via PachCTL
Features Overview
Enterprise Server (ES)
Activate ES for Multi-Cluster
Activate ES for Single-Cluster
Register a Cluster via Helm
Register a Cluster via PachCTL
Server Management
Server Setup
S3 Gateway API
TLS (SSL, HTTPS)
Tracing (Jaeger)
Blob/Object Storage
Global Config
Manage
Helm Chart Values (HCVs)
Deploy Target HCVs
Global HCVs
Console HCVs
Enterprise Server HCVs
ETCD HCVs
Ingress HCVs
Loki HCVs
PachD HCVs
PachW HCVs
Kube Event Tail HCVs
PGBouncer HCVs
PostgreSQL Subchart HCVs
CloudSQL Auth Proxy HCVs
OpenID Connect HCVs
Test Connection HCVs
Proxy HCVs
Determined HCVs
S3 Gateway
AWS CLI
Boto3
Credentials
MinIO
Unsupported Operations
Backup & Restore
Cluster Backup
Enterprise Server Backup
Upgrade
PachCTL Shell
Check IdP User
Supported Releases & Features
PostgresSQL Fine-Tuning
Cluster Access
Deactivate Authorization
GPUs
Log In via IdP
Revoke User Access
Scaling Limits (CE)
Secrets
Sidecar S3 Gateway
Storage Optimization
Usage Metrics
Monitor with Prometheus
Metrics
Uninstall
Prepare Data
Datum Batching
Defer Processing via Staging Branch
Ingest Data
Mount Volumes
Skip Failed Datums
Time-Windowed Data
Transactions
Build Pipelines & DAGs
Pipeline Specification (PPS)
Autoscaling PPS
Datum Set Spec PPS
Datum Timeout PPS
Datum Tries PPS
Description PPS
Egress PPS
Input Cron PPS
Input Cross PPS
Input Group PPS
Input Join PPS
Input PFS PPS
Input Union PPS
Job Timeout PPS
Metadata PPS
Output Branch PPS
Parallelism Spec PPS
Pod Patch PPS
Pod Spec PPS
Reprocess Spec PPS
Resource Limits PPS
Resource Requests PPS
s3 Out PPS
Scheduling Spec PPS
Service PPS
Sidecar Resource Limits PPS
Sidecar Resource Requests PPS
Spec Commit PPS
Spout PPS
Tolerations PPS
Transform PPS
Full Pipeline Specification
Pipeline Ops
Create a Pipeline
Delete a Pipeline
Draw a Pipeline
Inspect a Pipeline
Jsonnet Pipeline Specifications
Update a Pipeline
View Pipeline Jobs & Runtimes
Project Ops
Create a Project
Set a Project as Current
Add a Project Resource
Grant Project Access
Delete a Project
Branch Ops
Copy Files
Process Specific Commits
Set Branch Triggers
Set Output Branch
Datum Ops
Get Metadata
Inspect Datum
Provenance Ops
List Global Commits & Jobs
List Global ID Sub Commits
Track Downstream
Delete Branch Head
Squash Non-Head Commits
Delete File From History
Tutorials
Standard ML Pipeline
AutoML Pipeline
Multi-Pipeline DAG
Data Parallelism Pipeline
Task Parallelism Pipeline
Docker Image + User Code
Export Data
Egress To An SQL Database
Export via Egress
Export via PachCTL
Mount a Repo Locally
S3 Gateway Operations
Create S3 Bucket
Delete an S3 Object
Delete Empty S3 Bucket
Get an S3 Object
List S3 Buckets
List S3 Objects
Write an S3 Object
Integrate
Determined
Google BigQuery
JupyterLab
Docker Installation Guide
Run in Determined
Local Installation Guide
User Guide
Troubleshooting
Label Studio
Superb AI
VS Code Auto-Completion
Weights and Biases
Run Commands
pachctl
pachctl auth
pachctl auth activate
pachctl auth check
pachctl auth check project
pachctl auth check repo
pachctl auth deactivate
pachctl auth get
pachctl auth get cluster
pachctl auth get enterprise
pachctl auth get project
pachctl auth get repo
pachctl auth get-config
pachctl auth get-groups
pachctl auth get-robot-token
pachctl auth login
pachctl auth logout
pachctl auth revoke
pachctl auth roles-for-permission
pachctl auth rotate-root-token
pachctl auth set
pachctl auth set cluster
pachctl auth set enterprise
pachctl auth set project
pachctl auth set repo
pachctl auth set-config
pachctl auth use-auth-token
pachctl auth whoami
pachctl buildinfo
pachctl check
pachctl check status
pachctl completion
pachctl completion bash
pachctl completion zsh
pachctl config
pachctl config delete
pachctl config delete context
pachctl config get
pachctl config get active-context
pachctl config get active-enterprise-context
pachctl config get context
pachctl config get metrics
pachctl config import-kube
pachctl config list
pachctl config list context
pachctl config set
pachctl config set active-context
pachctl config set active-enterprise-context
pachctl config set context
pachctl config set metrics
pachctl config update
pachctl config update context
pachctl connect
pachctl copy
pachctl copy file
pachctl create
pachctl create branch
pachctl create defaults
pachctl create pipeline
pachctl create project
pachctl create repo
pachctl create secret
pachctl debug
pachctl debug analyze
pachctl debug binary
pachctl debug dump
pachctl debug local
pachctl debug log-level
pachctl debug profile
pachctl debug template
pachctl delete
pachctl delete all
pachctl delete branch
pachctl delete commit
pachctl delete commitV2
pachctl delete defaults
pachctl delete file
pachctl delete job
pachctl delete pipeline
pachctl delete project
pachctl delete repo
pachctl delete secret
pachctl delete transaction
pachctl diff
pachctl diff file
pachctl draw
pachctl draw pipeline
pachctl edit
pachctl edit pipeline
pachctl enterprise
pachctl enterprise deactivate
pachctl enterprise get-state
pachctl enterprise heartbeat
pachctl enterprise pause
pachctl enterprise pause-status
pachctl enterprise register
pachctl enterprise sync-contexts
pachctl enterprise unpause
pachctl exit
pachctl find
pachctl find commit
pachctl finish
pachctl finish commit
pachctl finish transaction
pachctl fsck
pachctl get
pachctl get file
pachctl glob
pachctl glob file
pachctl idp
pachctl idp create-client
pachctl idp create-connector
pachctl idp delete-client
pachctl idp delete-connector
pachctl idp get-client
pachctl idp get-config
pachctl idp get-connector
pachctl idp list-client
pachctl idp list-connector
pachctl idp set-config
pachctl idp update-client
pachctl idp update-connector
pachctl inspect
pachctl inspect branch
pachctl inspect cluster
pachctl inspect commit
pachctl inspect datum
pachctl inspect defaults
pachctl inspect file
pachctl inspect job
pachctl inspect pipeline
pachctl inspect project
pachctl inspect repo
pachctl inspect secret
pachctl inspect transaction
pachctl kube-events
pachctl license
pachctl license activate
pachctl license add-cluster
pachctl license delete-all
pachctl license delete-cluster
pachctl license get-state
pachctl license list-clusters
pachctl license update-cluster
pachctl list
pachctl list branch
pachctl list commit
pachctl list datum
pachctl list file
pachctl list job
pachctl list pipeline
pachctl list project
pachctl list repo
pachctl list secret
pachctl list transaction
pachctl logs
pachctl loki
pachctl mount
pachctl port-forward
pachctl put
pachctl put file
pachctl rerun
pachctl rerun pipeline
pachctl restart
pachctl restart datum
pachctl resume
pachctl resume transaction
pachctl run
pachctl run cron
pachctl run pfs-load-test
pachctl run pps-load-test
pachctl shell
pachctl squash
pachctl squash commit
pachctl squash commitV2
pachctl start
pachctl start commit
pachctl start pipeline
pachctl start transaction
pachctl stop
pachctl stop job
pachctl stop pipeline
pachctl stop transaction
pachctl subscribe
pachctl subscribe commit
pachctl unmount
pachctl update
pachctl update defaults
pachctl update pipeline
pachctl update project
pachctl update repo
pachctl validate
pachctl validate pipeline
pachctl version
pachctl wait
pachctl wait commit
pachctl wait job
Debug
Common Issues
Debug Pipelines
Troubleshooting Deployments
View Audit Logs
View Kubernetes Logs
Pachyderm SDK
Client Initialization (Start Here)
Method Mapping
First Project
Examples
Breast Cancer Detection
Distributed Image Processing
Spout Pipelines
Reference Docs
Contribute
Coding Conventions
Contributor Setup
Developing on Windows with VSCode
Documentation Style Guide
Home
Readme)
👉