Skip to content

Vertex Pipelines Deployer


Deploy Vertex Pipelines within minutes

This tool is a wrapper around kfp and google-cloud-aiplatform that allows you to check, compile, upload, run and schedule Vertex Pipelines in a standardized manner.


Info

This project is looking for beta testers and contributors.

You can contact code owners or submit a new issue if you want to help.

Why this tool?

Three use cases:

  1. CI: Check pipeline validity.
  2. Dev mode: Quickly iterate over your pipelines by compiling and running them in multiple environments (test, dev, staging, etc.) without duplicating code or searching for the right kfp/aiplatform snippet.
  3. CD: Deploy your pipelines to Vertex Pipelines in a standardized manner in your CD with Cloud Build or GitHub Actions.

Two main commands:

  • check: Check your pipelines (imports, compile, check configs validity against pipeline definition).
  • deploy: Compile, upload to Artifact Registry, run, and schedule your pipelines.

Installation

From PyPI

pip install vertex-deployer

From git repo

Stable version: 

pip install git+https://github.com/artefactory/vertex-pipelines-deployer.git@main

Develop version: 

pip install git+https://github.com/artefactory/vertex-pipelines-deployer.git@develop

If you want to test this package on examples from this repo: 

git clone git@github.com:artefactory/vertex-pipelines-deployer.git
poetry install
poetry shell  # if you want to activate the virtual environment
cd example

Basic Usage

Please refer to the Basic Usage page for more information.

License

This project is licensed under the terms of the Apache 2.0.