Deploy OCI Artifacts And Helm Charts The Gitops Way With Config Sync

One principle of GitOps is to have the desired state declarations as Versioned and Immutable, where Git repositories play an important role as the source of truth. But can you have an alternative to a Git repository for storing and deploying your Kubernetes manifests via GitOps? What if you could package your Kubernetes manifests into a container image instead? What if you can reuse the same authentication and authorization mechanism as your container images?

To answer the above questions, an understanding of OCI registries and OCI artifacts is needed. Simply put, OCI registries are the registries that are typically used for container images but can be expanded to store other types of data (aka OCI artifacts) such as Helm charts, Kubernetes manifests, Kustomize overlays, scripts, etc.

Partner with aster.cloud
for your next big idea.
Let us know here.

Using OCI Registries and OCI Artifacts provides you with the following advantages:

  • Less tools to operate: Single artifact registry can store expanded data types apart from container images.
  • In-built release archival system: OCI registries give users two sets of mutable and immutable URLs which are tags and content-addressable ones.
  • Flourishing ecosystem: Standardized and supported by dozen of providers which helps users take advantage of new features and tools developed by large Kubernetes community

Given these benefits, and in addition to the support of files stored in Git repositories, we are thrilled to announce two new formats supported by Config Sync 1.13 to deploy OCI artifacts:

 

Config Sync is an open source tool that provides GitOps continuous delivery for Kubernetes clusters.

The Open Container Initiative (OCI) is an open governance structure for the express purpose of creating open industry standards around container formats and runtimes. OCI artifacts give you the power of storing and distributing different types of data such as Kubernetes manifests, Helm Charts, and Kustomize overlays, in addition to container images via OCI registries.

Throughout this blog, you will see how you can leverage the two new formats (OCI artifacts and Helm charts) supported by Config Sync, by using:

  • oras and helm to package and push OCI artifacts
  • Artifact registry as OCI registry to store the OCI artifacts
  • GKE cluster to host the OCI artifacts synced
  • Config Sync installed in that GKE cluster to sync the OCI artifacts
Read More  FEDRAMP High Development in the Cloud: Code with Cloud Workstations

Initial setup

First, you need to have a common setup for the two scenarios by configuring and securing the access from the GKE cluster with Config Sync to the Artifact Registry repository.

 

Initialize the Google Cloud project you will use throughout this blog:

 

PROJECT=SET_YOUR_PROJECT_ID_HERE
gcloud config set project $PROJECT

 

Create a GKE cluster with Workload Identity registered in a fleet to enable Config Management:

 

CLUSTER_NAME=oci-artifacts-cluster
REGION=us-east4
gcloud services enable container.googleapis.com
gcloud container clusters create ${CLUSTER_NAME} \
    --workload-pool=${PROJECT}.svc.id.goog \
    --region ${REGION}
gcloud services enable gkehub.googleapis.com
gcloud container fleet memberships register ${CLUSTER_NAME} \
    --gke-cluster ${REGION}/${CLUSTER_NAME} \
    --enable-workload-identity
gcloud beta container fleet config-management enable

 

Install Config Sync in the GKE cluster:

 

cat <<EOF > acm-config.yaml
applySpecVersion: 1
spec:
  configSync:
    enabled: true
EOF
gcloud beta container fleet config-management apply \
    --membership ${CLUSTER_NAME} \
    --config acm-config.yaml

 

Create an Artifact Registry repository to host OCI artifacts (--repository-format docker):

 

CONTAINER_REGISTRY_NAME=oci-artifacts
gcloud services enable artifactregistry.googleapis.com
gcloud artifacts repositories create ${CONTAINER_REGISTRY_NAME} \
    --location ${REGION} \
    --repository-format docker

 

Create a dedicated Google Cloud Service Account with the fine granular access to that Artifact Registry repository with the roles/artifactregistry.reader role:

 

GSA_NAME=oci-artifacts-reader
gcloud iam service-accounts create ${GSA_NAME} \
    --display-name ${GSA_NAME}
gcloud artifacts repositories add-iam-policy-binding ${CONTAINER_REGISTRY_NAME} \
    --location ${REGION} \
    --member "serviceAccount:${GSA_NAME}@${PROJECT}.iam.gserviceaccount.com" \
    --role roles/artifactregistry.reader

 

Allow Config Sync to synchronize resources for a specific RootSync:

 

ROOT_SYNC_NAME=root-sync-oci
gcloud iam service-accounts add-iam-policy-binding \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:${PROJECT}.svc.id.goog[config-management-system/root-reconciler-${ROOT_SYNC_NAME}]" \
    ${GSA_NAME}@${PROJECT}.iam.gserviceaccount.com

 

Login to Artifact Registry so you can push OCI artifacts to it in a later step:

 

gcloud auth configure-docker ${REGION}-docker.pkg.dev

 

Build and sync an OCI artifact

Now that you have completed your setup, let’s illustrate our first scenario where you want to sync a Namespace resource as an OCI image.

 

Create a Namespace resource definition:

Read More  Google Cloud Next 2019 | Comprehensive Protection of PII in GCP

 

cat <<EOF> test-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: test
EOF

 

Create an archive of that file:

 

tar -cf test-namespace.tar test-namespace.yaml

 

Push that artifact to Artifact Registry. In this tutorial, we use oras, but there are other tools that you can use like crane.

 

oras push \
    ${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}/my-namespace-artifact:v1 \
    test-namespace.tar

 

Set up Config Sync to deploy this artifact from Artifact Registry:

 

cat << EOF | kubectl apply -f -
apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
  name: ${ROOT_SYNC_NAME}
  namespace: config-management-system
spec:
  sourceFormat: unstructured
  sourceType: oci
  oci:
    image: ${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}/my-namespace-artifact:v1
    dir: .
    auth: gcpserviceaccount
    gcpServiceAccountEmail: ${GSA_NAME}@${PROJECT}.iam.gserviceaccount.com
EOF

 

Check the status of the sync with the nomos tool:

 

nomos status --contexts $(k config current-context)

 

Verify that the Namespace test is synced:

 

kubectl get ns test

 

And voilà! You just synced a Namespace resource as an OCI artifact with Config Sync.

Build and sync a Helm chart

Now, let’s see how you could deploy a Helm chart hosted in a private Artifact Registry.

 

Create a simple Helm chart:

 

helm create test-chart

 

Package the Helm chart:

 

helm package test-chart --version 0.1.0

 

Push the chart to Artifact Registry:

 

helm push \
    test-chart-0.1.0.tgz \
    oci://${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}

 

Set up Config Sync to deploy this Helm chart from Artifact Registry:

 

cat << EOF | kubectl apply -f -
apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
  name: ${ROOT_SYNC_NAME}
  namespace: config-management-system
spec:
  sourceFormat: unstructured
  sourceType: helm
  helm:
    repo: oci://${REGION}-docker.pkg.dev/${PROJECT}/${CONTAINER_REGISTRY_NAME}
    chart: test-chart
    version: 0.1.0
    releaseName: test-chart
    namespace: default
    auth: gcpserviceaccount
    gcpServiceAccountEmail: ${GSA_NAME}@${PROJECT}.iam.gserviceaccount.com
EOF

 

Check the status of the sync with the nomos tool:

 

nomos status --contexts $(k config current-context)

 

Verify that the resources in the Namespace default are synced:

 

kubectl get all -n default

 

And voilà! You just synced an Helm chart with Config Sync.

Towards more scalability and security

In this blog, you synced both an OCI artifact and an Helm chart with Config Sync.

OCI registries and OCI artifacts are new kids on the block that can also work alongside with the Git option depending on your needs and use-cases. One of such patterns could be Git still acting as the source of truth for the declarative configs in addition to the well established developer workflow it provides: pull request, code review, branch strategy, etc.

Read More  Introducing Query Insights For Cloud Spanner: Troubleshoot Performance Issues With Pre-Built Dashboards

The continuous integration pipelines, triggered by pull requests or merges, will run tests against the declarative configs to eventually push the OCI artifacts in an OCI registry.

Finally, the continuous reconciliation of GitOps will take it from here and will reconcile between the desired state, now stored in an OCI registry, with the actual state, running in Kubernetes. Your Kubernetes manifests as OCI artifacts are now just seen like any container images for your Kubernetes clusters as they are pulled from OCI registries. This continuous reconciliation from OCI registries, not interacting with Git, has a lot of benefits in terms of scalability, performance and security as you will be able to configure very fine grained access to your OCI artifacts.

Next steps

To get started, check out the two Sync OCI artifacts from Artifact Registry and the Sync Helm charts from OCI registries features today.

You can also leverage these two following tutorials illustrating how to do CI/GitOps with Helm charts from GitHub Actions to Config Sync:

Attending KubeCon + CloudNativeCon North America 2022 in October? Come check out our session Build and Deploy Cloud Native (OCI) Artifacts, the GitOps Way during the GitOpsCon North America 2022 co-located event on October, 25th. Hope to see you there!

Config Sync is open sourced. We are open to contributions and bug fixes if you want to get involved in the development of Config Sync. You can also use the repository to track ongoing work, or build from source to try out bleeding-edge functionalities.

 

 

By: Mathieu Benoit (DevRel Engineer) and Divyansh Chaturvedi (Product Manager)
Source: Google Cloud Blog

For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!
Our humans need coffee too! Your support is highly appreciated, thank you!

Related Topics
You May Also Like