aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Computing
  • Engineering

How To Migrate A Group Of Individual Instances To A Stateful MIG Using Python Script

  • aster.cloud
  • July 11, 2022
  • 4 minute read

A GCP Compute Engine managed instance group (MIG) supports any VM configuration that you need, and it helps manage the VMs for you. For example, when a VM in a MIG unexpectedly stops running, the MIG recreates that VM according to the configuration that you set. You can also set up an application-based health check to verify that your application responds as expected on each VM.

In addition to that, MIGs also allows you to deploy complex stateful applications, such as databases (Cassandra, ElasticSearch) or data processing applications (Kafka, Flink), where preservation of individual VM state (for example, a database shard, or app configuration) is important.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

If you are just starting to work with GCP and want to deploy a stateful application, then it’s highly recommended to use a stateful MIG configuration. A stateful MIG allows you to preserve the unique state of each instance (including instance name, attached persistent disks, IP addresses – available in Preview, and metadata) on VM restart, recreation, auto-healing, or update. You can learn more about Stateful MIGs by reading the docs.

The goal of this blog post is to help those who already have an existing stateful application running on standalone (unmanaged) Compute Engine instances and who want to automatically migrate them to a stateful MIG using a Python script. The script code and detailed instructions for its use and installation can be found here. In case you want to do this whole process manually, you can go through the Migrate an existing workload to a stateful managed instance group tutorial.

Let’s dive into the logic of the script and see how it can be executed.

Read More  3 Common Myths About Sustainability And Cloud Computing

Script requirements

Before running the script, make sure your existing instances meet the following criteria:

  • All source instances must have the same instance configuration (for example, machine type). Remember that all instances in a MIG are created according to the same instance template. The script creates a template for you based on one of your existing instances.
  • Your boot disks should be stateless. In case you must preserve state on your boot disks, then you can still create a stateful MIG, but you cannot update operating systems or software by rolling out boot image updates.
  • The script will stop your source VM instances. The script leaves all source VMs stopped with their disks intact, for easy reverting if the MIG doesn’t work as expected. This results in additional costs, for the following reasons:
  1. The script doesn’t detach or delete the original disks.
  2. The script creates images from existing disks.
  • You do not depend on your VM names. Source VM names remain unchanged. The new VMs in the MIG will have new names. Pay attention to this if you are somehow tied to the names of your VMs.

Script steps

  1. Stop all instances
  2. If needed, create a boot disk image
  3. Create an instance template based on the properties of a chosen instance, while ignoring attached data disks.
  4. Create an empty MIG.
  5. For each instance in the original group, perform the following steps:
    1. Clone all instance disks except the boot disk.
    2. Create an instance in the MIG based on the instance template, and include the cloned disks from the source instance.
  6. Print commands for cleaning up the source instances after you have verified that the stateful MIG serves your needs.
Read More  Introducing New Cloud Client Libraries For Compute Engine

If you are ready to use the script, follow the link for installation and execution.

Usage example

Let’s now imagine some instance configuration and show how the script proceeds. Suppose we have three standalone instances with names “alpha”, “beta”, and “gamma” in the us-central1-a zone and we want to migrate them to a MIG with the name “greek”.

Instance “alpha” has two additional disks: “alpha-disk-1” and “alpha-disk-2”, instance “beta” has one additional disk “beta-disk-1”, and “gamma” has zero additional disks. At the same time we would like to create a disk image for the boot image.

Run the following command:

$ python3 migrate_script.py -s alpha beta gamma -z us-central1-a -m greek --image_for_boot_disk

Let’s take a look on the output step by step:

  1. All the instances are stopped at the very beginning.

 

Instance alpha is not stopped. Stopping …
Instance alpha stopped

==========

Instance beta is not stopped. Stopping ...

Instance beta stopped

==========

Instance gamma is not stopped. Stopping ...

Instance gamma stopped

==========

 

2. Because we set the “image_for_boot_disk” flag, an image for the boot disk is created.

 

Creating disk image for boot image alpha …
Disk image alpha-image-ed0d24 created
==========

 

3. By default, alpha is chosen to specify machine configuration in the instance template (it’s possible to set another instance explicitly).

 

Creating base instance template ...
Instance template alpha-template-004cb8 created
==========

 

4. An empty MIG “gamma” is created.

 

Creating empty MIG gamma...
MIG gamma created
==========

 

5. For each individual instance, we create copies of all its disks (not to damage source instances data) and attach them to VM in the MIG.

Read More  How To Help Ensure Smooth Shift Handoffs In Security Operations

 

Creating disk alpha-disk-1-bb2a30 from disk alpha-disk-1
==========
Creating disk alpha-disk-2-f7344b from disk alpha-disk-2
==========
Adding instance alpha-b7273d to greek MIG
==========
Creating disk beta-disk-1--7498a2 from disk beta-disk-1
==========
Adding instance beta-6475c6 to greek MIG
==========
Adding instance gamma-17d7c7 to greek MIG
==========
Migration successfully finished. Time spent: 457 seconds.

 

If you are convinced that everything is fine with the newly created MIG, you can delete your source instances. Use the following command:

 

$ gcloud compute instances delete alpha beta gamma

 

If instead, for any reason, you want to return to your initial configuration, use the following clean-up commands in the terminal or in the Cloud Shell:

 

$ gcloud compute instance-groups managed delete greek
$ gcloud compute instance-templates delete alpha-template-004cb8
$ gcloud compute disks delete alpha-disk-1-bb2a30
$ gcloud compute disks delete alpha-disk-2-f7344b
$ gcloud compute disks delete beta-disk-1-7498a2
$ gcloud compute images delete alpha-image-ed0d24

 

Congratulations! Your MIG is set up! What’s next?

– Configure autohealing

– Use a stateful policy instead of per-instance configurations

– Add more VMs

 

 

By: Fedor Isakov (Developer Relations Engineer)
Source: Google Cloud Blog


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Compute Engine
  • Google Cloud
  • Python
  • Tutorials
You May Also Like
View Post
  • Engineering

Just make it scale: An Aurora DSQL story

  • May 29, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Reliance on US tech providers is making IT leaders skittish

  • May 28, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Examine the 4 types of edge computing, with examples

  • May 28, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

AI and private cloud: 2 lessons from Dell Tech World 2025

  • May 28, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

TD Synnex named as UK distributor for Cohesity

  • May 28, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more

  • May 28, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Weigh these 6 enterprise advantages of storage as a service

  • May 28, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Pulsant targets partner diversity with new IaaS solution

  • May 23, 2025

Stay Connected!
LATEST
  • 1
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 2
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • 5
    TD Synnex named as UK distributor for Cohesity
    • May 28, 2025
  • Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • 7
    Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more
    • May 28, 2025
  • 8
    Pulsant targets partner diversity with new IaaS solution
    • May 23, 2025
  • 9
    Growing AI workloads are causing hybrid cloud headaches
    • May 23, 2025
  • Gemma 3n 10
    Announcing Gemma 3n preview: powerful, efficient, mobile-first AI
    • May 22, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • Understand how Windows Server 2025 PAYG licensing works
    • May 20, 2025
  • By the numbers: How upskilling fills the IT skills gap
    • May 21, 2025
  • 3
    Cloud adoption isn’t all it’s cut out to be as enterprises report growing dissatisfaction
    • May 15, 2025
  • 4
    Hybrid cloud is complicated – Red Hat’s new AI assistant wants to solve that
    • May 20, 2025
  • 5
    Google is getting serious on cloud sovereignty
    • May 22, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.