GKE Release Channels: Balancing Innovation And Speed Of Change, Now With More Granular Controls

If you run your business on Kubernetes, you know how important it is to perform regular patching and upgrades to maintain a healthy environment. At Google Cloud, we automatically upgrade the Google Kubernetes Engine (GKE) cluster control plane, and if you enable node auto-upgrade, we also automatically upgrade and patch your cluster nodes. Moreover, you can  subscribe your cluster to a release channel that meets your constraints and needs — rapid, regular or stable. This is just one of the ways that GKE supports enterprises running production workloads, and is broadly used by GKE customers as part of their continuous upgrade strategy. For enterprises, release channels provide the level of predictability needed for advanced planning, and the flexibility to orchestrate custom workflows automatically when a change is scheduled (e.g. informing their DevOps team when a new security patch is available).

While automating upgrades with release channels simplifies keeping track of Kubernetes versions and OSS compatibility, you may want to upgrade your cluster at specific times, for business continuity or quality assurance reasons. You may also want to control the scope of the upgrades —  apply just patches, or avoid minor releases — allowed on your cluster. This is critical, especially when you feel that an upgrade requires more qualification, testing, or preparation before it can be rolled out to production.

We recently enhanced the maintenance windows that you can set in GKE release channels. Previously, maintenance windows allowed you to specify “no upgrades” to the control plane or nodes for up to 30 days. Now, the upgrade-scope exclusion allows you to control and limit the scope of upgrades for up to 180 days, or end-of-life date, whichever comes first. Likewise, you can preclude minor upgrades and node upgrades from being applied to both your control planes and nodes with two new modes, “no_minor_upgrades” and “no_minor_or_nodes_upgrades”. And in both cases, you can “pin” your cluster to a specific minor k8s version (say 1.21) for a prolonged period of up to six months.

GKE release channels in action

German food-delivery network Delivery Hero is one GKE customer that recently began using release channels. With 791 million orders processed in the third quarter of 2021, Delivery Hero initially chose to eliminate potential disruptions to its customers by relying on a manual process to control the timing of changes, and reduce the risk that an untested update might impact availability. But this was not an ideal solution: constantly monitoring the Kubernetes release schedule, tracking version skew compatibility, and applying security patches was cumbersome.

In an effort to balance between risk mitigation and operational efficiency, Delivery Hero decided to subscribe their GKE clusters to a release channel. But in order to do it even more safely, they also defined the scope of auto-upgrades to include only patch versions, and to postpone minor upgrades. This way their GKE cluster is patched automatically to ensure security and compliance, but they hold back on minor version upgrades until they can be internally tested and qualified.

“Before the option to control upgrade scope, and especially the ability to postpone minor upgrades up to 6 months, we were struggling to align our qualification timeline with the cadence of Kubernetes OSS releases, especially with the API deprecations in recent releases,” said Kumar Saurabh Sinha, Engineering Manager (Platform & SRE) at Delivery Hero. “With upgrade scope exclusions, we managed to safely migrate our clusters and subscribe them on release channels while still having the ability to mitigate the risk of untested minor releases.”

Get started with GKE release channels today. When you create a new GKE cluster, it is automatically subscribed to the ‘regular’ release channel by default.

You can also migrate existing clusters onto release channels from the Google Cloud Console, or the command line. Read more here.

For example, if you want to subscribe a cluster to a release channel, and also to avoid minor Kubernetes upgrades for three months. You can follow these steps:

• Ensure that clusters are running a version supported by the channel.
• Exclude auto-upgrade for 24 hours. This is an optional safety step to avoid unplanned upgrades immediately after subscribing the cluster to a channel.
• Subscribe the clusters to your desired release channel.
• Set upgrade scope to no_minor_upgrades, allowing only patch versions to be applied to the cluster, while keeping the cluster on the same minor release.

With GKE release channels, you have the power to decide not only when, but how, and what to upgrade in your clusters and nodes. You can learn more about release channels here, and about maintenance windows here. And for even more, tune into this episode of the Google Cloud Platform Podcast, where my colleague Abdelfettah Sghiouar and I discuss the past and future of GKE release channels with the hosts Kaslin Fields and Mark Mirchandani.