A GCP Compute Engine managed instance group (MIG) supports any VM configuration that you need, and it helps manage the VMs for you. For example, when a VM in a MIG unexpectedly stops running, the MIG recreates that VM according to the configuration that you set. You can also set up an application-based health check to verify that your application responds as expected on each VM.
In addition to that, MIGs also allows you to deploy complex stateful applications, such as databases (Cassandra, ElasticSearch) or data processing applications (Kafka, Flink), where preservation of individual VM state (for example, a database shard, or app configuration) is important.
From our partners:
If you are just starting to work with GCP and want to deploy a stateful application, then it’s highly recommended to use a stateful MIG configuration. A stateful MIG allows you to preserve the unique state of each instance (including instance name, attached persistent disks, IP addresses – available in Preview, and metadata) on VM restart, recreation, auto-healing, or update. You can learn more about Stateful MIGs by reading the docs.
The goal of this blog post is to help those who already have an existing stateful application running on standalone (unmanaged) Compute Engine instances and who want to automatically migrate them to a stateful MIG using a Python script. The script code and detailed instructions for its use and installation can be found here. In case you want to do this whole process manually, you can go through the Migrate an existing workload to a stateful managed instance group tutorial.
Let’s dive into the logic of the script and see how it can be executed.
Script requirements
Before running the script, make sure your existing instances meet the following criteria:
- All source instances must have the same instance configuration (for example, machine type). Remember that all instances in a MIG are created according to the same instance template. The script creates a template for you based on one of your existing instances.
- Your boot disks should be stateless. In case you must preserve state on your boot disks, then you can still create a stateful MIG, but you cannot update operating systems or software by rolling out boot image updates.
- The script will stop your source VM instances. The script leaves all source VMs stopped with their disks intact, for easy reverting if the MIG doesn’t work as expected. This results in additional costs, for the following reasons:
- The script doesn’t detach or delete the original disks.
- The script creates images from existing disks.
- You do not depend on your VM names. Source VM names remain unchanged. The new VMs in the MIG will have new names. Pay attention to this if you are somehow tied to the names of your VMs.
Script steps
- Stop all instances
- If needed, create a boot disk image
- Create an instance template based on the properties of a chosen instance, while ignoring attached data disks.
- Create an empty MIG.
- For each instance in the original group, perform the following steps:
- Clone all instance disks except the boot disk.
- Create an instance in the MIG based on the instance template, and include the cloned disks from the source instance.
- Print commands for cleaning up the source instances after you have verified that the stateful MIG serves your needs.
If you are ready to use the script, follow the link for installation and execution.
Usage example
Let’s now imagine some instance configuration and show how the script proceeds. Suppose we have three standalone instances with names “alpha”, “beta”, and “gamma” in the us-central1-a zone and we want to migrate them to a MIG with the name “greek”.
Instance “alpha” has two additional disks: “alpha-disk-1” and “alpha-disk-2”, instance “beta” has one additional disk “beta-disk-1”, and “gamma” has zero additional disks. At the same time we would like to create a disk image for the boot image.
Run the following command:
$ python3 migrate_script.py -s alpha beta gamma -z us-central1-a -m greek --image_for_boot_disk
Let’s take a look on the output step by step:
- All the instances are stopped at the very beginning.
Instance alpha is not stopped. Stopping …
Instance alpha stopped
==========
Instance beta is not stopped. Stopping ...
Instance beta stopped
==========
Instance gamma is not stopped. Stopping ...
Instance gamma stopped
==========
2. Because we set the “image_for_boot_disk” flag, an image for the boot disk is created.
Creating disk image for boot image alpha …
Disk image alpha-image-ed0d24 created
==========
3. By default, alpha is chosen to specify machine configuration in the instance template (it’s possible to set another instance explicitly).
Creating base instance template ...
Instance template alpha-template-004cb8 created
==========
4. An empty MIG “gamma” is created.
Creating empty MIG gamma...
MIG gamma created
==========
5. For each individual instance, we create copies of all its disks (not to damage source instances data) and attach them to VM in the MIG.
Creating disk alpha-disk-1-bb2a30 from disk alpha-disk-1
==========
Creating disk alpha-disk-2-f7344b from disk alpha-disk-2
==========
Adding instance alpha-b7273d to greek MIG
==========
Creating disk beta-disk-1--7498a2 from disk beta-disk-1
==========
Adding instance beta-6475c6 to greek MIG
==========
Adding instance gamma-17d7c7 to greek MIG
==========
Migration successfully finished. Time spent: 457 seconds.
If you are convinced that everything is fine with the newly created MIG, you can delete your source instances. Use the following command:
$ gcloud compute instances delete alpha beta gamma
If instead, for any reason, you want to return to your initial configuration, use the following clean-up commands in the terminal or in the Cloud Shell:
$ gcloud compute instance-groups managed delete greek
$ gcloud compute instance-templates delete alpha-template-004cb8
$ gcloud compute disks delete alpha-disk-1-bb2a30
$ gcloud compute disks delete alpha-disk-2-f7344b
$ gcloud compute disks delete beta-disk-1-7498a2
$ gcloud compute images delete alpha-image-ed0d24
Congratulations! Your MIG is set up! What’s next?
– Use a stateful policy instead of per-instance configurations
By: Fedor Isakov (Developer Relations Engineer)
Source: Google Cloud Blog
For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!
Our humans need coffee too! Your support is highly appreciated, thank you!