Add GitOps Without Throwing Out Your CI Tools

Many teams are hesitant to introduce changes to their CI pipeline to not disrupt the often complex continuous deployment (CD) patterns. GitOps however is more than CD, it provides automation, eliminates unpredictability of configuration drift and enhances security.

Many Continuous Integration (CI) tools claim to do GitOps but what they actually do is Continuous Deployment (CD). Continuous Deployment is not GitOps but GitOps does provide Continuous Deployment and so much more. If you want to do true GitOps, do you have to deprecate all the work you’ve done building out your CI tooling? Read on to find out.

CD is not GitOps

Continuous Deployment (CD) is not GitOps. To be true GitOps the following principles as defined by OpenGitOps must be followed.

• Declarative – A system managed by GitOps must have its desired state expressed declaratively.
• Versioned and Immutable – Desired state is stored in a way that enforces immutability, versioning and retains a complete version history.
• Pulled Automatically – Software agents automatically pull the desired state declarations from the source.
• Continuously Reconciled – Software agents continuously observe actual system state and attempt to apply the desired state.

Continuous Integration (CI) tools easily conform to the first two points, it’s points three and four where they fall short. CI tools typically take a push approach and lack any form of feedback; push and forget.

Push vs Pull

A Continuous Integration pipeline is typically triggered by a merge into the main branch of a Git repository; some form of webhook is called to trigger the start of the pipeline. The repository is cloned into the build system, artefacts are built, pushed to a registry and finally the runtime configuration is pushed to the target environment.

This approach works well until changes are directly made to the runtime environment, shortcutting Git and the CI/CD pipeline. The result is that the actual state, running in Kubernetes, no longer matches the desired state, defined in Git. You’ve lost control and configuration has started to drift. There’s no audit trail of who changed what, the changes have not been subject to Policy as Code checks nor peer review. Things are going to get chaotic.

GitOps works in a similar way to the above flow but with some important differences. It does not replace the Continuous Integration tooling, it follows on to provide the Continuous Deployment functionality; sort of CI/GitOps. Continuous Deployment with GitOps differs in that it automatically pulls changes and continually reconciles those changes; not push and forget.

First of all let’s look at a simple scenario where a configuration change is made without requiring the building of an artefact. The change is made in Git via a pull request, policy as code check and peer review. There will not be a trigger for the CI pipeline because nothing requires building. The GitOps agent running on the Kubernetes cluster notices the change merged into the main branch; no requirement to configure webhooks etc. The configuration change is automatically applied to the cluster.

Now let’s see what happens when there’s a source code change requiring an artefact to be built. As before the change is facilitated by the usual workflow of PR, review and merge. This time because the source code was modified, the CI pipeline will be triggered to build, test and push the artefact to the registry. The GitOps agent running on the Kubernetes cluster notices a new image tag available in the registry and updates the configuration in Git with the new image tag; this can be a direct merge into main or via a pull request. The agent applies the new configuration to the cluster resulting in the version being deployed.

The pull methodology used by GitOps provides higher levels of automation, reducing the burden on the DevOps team of defining and maintaining pipeline definitions.

Continuous Reconciliation

The differences between push and pull outlined in the previous section are perhaps subtle and ultimately both achieve the same result. I’ve saved the best ‘til last. Continuous Integration tools push the new configuration to the target environment, the pipeline finishes and that’s it. With GitOps the reconciliation between the desired state, stored in Git, with the actual state, running in Kubernetes, is continuous; big difference.

Taking the first scenario from the previous section, a simple configuration change. Sometime after the change was completed, either accidentally or maliciously a change is directly applied to the configuration of a Kubernetes entity. The CI tooling is oblivious, the pipeline has finished execution. GitOps, however, is continuously reconciling between desired and actual state, rather like Kubernetes does, the directly applied change affects the actual state, it no longer matches the desired state. GitOps will revert the directly applied change back to the desired state, zero configuration drift, order is maintained and chaos is averted. This significantly improves system maintainability, you know exactly how the environment is configured. There are implications for the security of the environment too, unauthorized changes are just not possible.

GitOps Completes your CI Pipelines

GitOps does not replace your CI pipelines, it enhances them. Providing greater automation, eliminating the unpredictability of configuration drift and enhancing security by preventing unauthorized changes. Retain your investment in your current Constant Integration tooling and use GitOps to provide Constant Deployment. It’s time to switch to CI/GitOps.

Guest post originally published on the Weaveworks blog
Source CNCF