aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • DevOps
  • Public Cloud

Simplify Your DevOps Using Skaffold

  • aster.cloud
  • April 5, 2022
  • 5 minute read

As the Public Sector Solutions team at Google Cloud, we have many opportunities to develop solutions that make the most of the cutting-edge Cloud technologies. In many projects, we collaborate with multiple development teams to develop, test, and deploy microservice-based solutions, often to GKE clusters.

In this blog, we will address these challenges of managing such complex development environments. We will also discuss using Skaffold and Kubernetes for repeatable development and the DevOps process for large development teams.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Complexity of Developing with Kubernetes

Microservices have become a common system architecture allowing for higher scalability and flexibility, and Kubernetes has become the go-to option for deploying  a complex microservice-based system. One of the benefits of microservices is that each individual service is standalone, self-encapsulated, and less dependent on other components in the system. It’s also easy to spin up a new microservice to meet a new requirement as a Kubernetes service.

Although the benefits of developing on Kubernetes are clear, it’s often difficult to design and implement an efficient and simple development process. This complexity increases greatly when a project involves a number of microservices managed by multiple teams.

Dedicated DevOps efforts and costs

It is critical for a successful project to start with a robust and reliable development process. This setup usually requires full-time DevOps or Platform engineers to design and develop a process to bootstrap the infrastructure, including environment configurations, building, testing, deployment, and CI/CD. These DevOps tasks not only take up a significant portion of the project in its early stage, but also are repetitive across new projects.

Similarly, development teams may not be able to avoid these DevOps tasks. A software engineer usually spends a significant amount of time handling the environment setup rather than feature development, e.g. tuning configurations to troubleshoot a microservice in a remote cluster with a close-to-production scenario.

Read More  How AI Can Improve Digital Security

Repetitive steps for a single deployment

Oftentime, engineers run various tools and commands just to deploy to either a local or a remote copy of the application. These steps in the development lifecycle are repetitive and often an extra tax on developer’s time. For example, to build and run a backend and a frontend microservices at the same time, an engineer would need to run commands like below:

$ docker build -t backend-tag backend
$ docker build -t frontend-tag frontend

$ kubectl apply -f backend/deploy.yaml
$ kubectl apply -f frontend/deploy.yaml

$ kubectl port-forward service/backend 8000
$ kubectl port-forward service/frontend 3000

When there’s any code changes, the engineer would need to rerun these commands to redeploy code to the Kubernetes pods to reflect the changes. If there are 10+ microservices to test with, things get more complicated. These steps may also be subject to change for different environments. For example, the steps for deploying microservices to a local minikube setup may be different from deploying to a GKE-based cluster.

This complexity slows down the development progress, and inevitably increases the overall cost.

Solution: Skaffold for all environments

In our solution development projects, we have developed a repeatable infrastructure using Skaffold to develop and run microservice-based solutions in GKE. Skaffold is a command line tool that facilitates continuous development for Kubernetes-native applications. Skaffold handles the workflow for building, pushing, and deploying your application, and provides building blocks for creating CI/CD pipelines. This enables you to focus on iterating on your application locally while Skaffold continuously deploys to your local or remote Kubernetes cluster.

We use an infrastructure template that encapsulates our best practices and most common designs can be easily copied and applied to any new project with the same structure using Skaffold, Kustomize and Kubernetes. In a nutshell, we use Skaffold to unify and simplify our development process with one command for all development and deployment environments.

Read More  Palexy Empowers Retailers To Increase In-Store Sales With The Help Of Google Cloud
Build & Deploy

 

  • Largely simplified local development process: In the local development environment, our engineers run Skaffold CLI command to build one or multiple microservices and deploy to either a local minikube setup or a remote cluster in GKE. They don’t need to worry about maintaining and modifying deployment scripts for different environments. All they need to do is to use Skaffold to build and test with desired services. The overall development process has become much more efficient than before, and engineers can focus on feature development rather than DevOps.
  • Streamlined deployment to all environments in GKE: All deployment to various environments including local, staging and production is unified with Skaffold using Profiles to differentiate the deployment configuration. Using Skaffold profiles not only simplifies the CI/CD configurations but also makes troubleshooting a lot easier.

Microservice-based structure with Skaffold and Kustomize

We have designed a generic project structure to streamline development with Skaffold. This structure is organized with microservices and components using Skaffold profiles and Kustomize. The overall file structure of a project looks like below:

 

Profile file structure

The project structure contains a “main Skaffold YAML” that defines the entire project, breaking it down with common components and microservices. In this root level, we define Skaffold profiles for various environments like Dev, Staging, Production or any custom environment. These predefined profiles cover all scenarios from local development to CI/CD deployment to remote cluster.

On the other hand, in each microservice folder we define the same set of profiles and use Kustomize to configure how to build and deploy in each environment. Each microservice can also define its preferred configuration including pods auto-scaling, CPU and memory settings, etc. See the example below.

Read More  One Year In, Konveyor Community Is Bringing Value To Clients’ Modernization Journeys
Microservice file structure

In addition, Skaffold has a built-in Watch Mode with “File Sync” feature to automatically live-reload changes in the local files and swap them to a remote cluster. This enables engineers to write code and see the changes immediately on a remote cluster.

Build

In general, engineers rely not only on Watch Mode but also Skaffold’s ability to stream logs while testing new code in a remote cluster in GKE. These features enable engineers to effectively live-troubleshoot issues in a remote cluster, as if it were a local cluster like minikube.

Live log streaming with a microservice deployed to a remote cluster.

All of these development workflow mentioned above can be done with a simple Skaffold command of “skaffold dev”.

Conclusion

Skaffold provides our teams with not only the benefits mentioned above, but also additional perks including debugging with breakpoints. With the solution infrastructure using Skaffold, we are able to effectively manage the development with multiple teams in collaboration, and we can easily expand the teams and projects to any scale using the same structure.

To summarize the achievements with Skaffold:

  • Development and deployment processes are unified and largely simplified.
  • Engineers spend very minimal time on DevOps, and more on feature development.
  • Skaffold enables the teams to quickly build and test with livereload and live-troubleshooting in a remote cluster.
  • The infrastructure is easy to replicate and expand to other projects.

If you haven’t started using Skaffold, we encourage you to check out the following Skaffold resources to get started:

  • Getting Started with Skaffold (Youtube Video)
  • https://skaffold.dev/docs/quickstart/
  • https://github.com/GoogleContainerTools/skaffold

 

 

By Jonathan Chen, Solutions Architect, Google Cloud | Drew Hodun, ML Solutions Architect, Google Cloud
Source Google Cloud


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Google Cloud
  • Kubernetes
  • Skaffold
  • Skaffold YAML
You May Also Like
View Post
  • Computing
  • Public Cloud
  • Technology

United States Army Enterprise Cloud Management Agency Expands its Oracle Defense Cloud Services

  • April 15, 2025
DeepSeek R1 is now available on Azure AI Foundry and GitHub
View Post
  • Public Cloud
  • Technology

DeepSeek R1 is now available on Azure AI Foundry and GitHub

  • February 2, 2025
Cloud platforms among the clouds
View Post
  • Computing
  • Learning
  • Public Cloud

Best Cloud Platforms Offering Free Trials for Cloud Mastery

  • December 23, 2024
Vehicle Manufacturing
View Post
  • Hybrid Cloud
  • Public Cloud

Toyota shifts into overdrive: Developing an AI platform for enhanced manufacturing efficiency

  • December 10, 2024
IBM and AWS
View Post
  • Public Cloud

IBM and AWS Accelerate Partnership to Scale Responsible Generative AI

  • December 2, 2024
COP29 AI and Climate Change
View Post
  • Public Cloud
  • Technology

How Cloud And AI Are Bringing Scale To Corporate Climate Mitigation And Adaptation

  • November 18, 2024
Cloud Workstations
View Post
  • Public Cloud

FEDRAMP High Development in the Cloud: Code with Cloud Workstations

  • November 8, 2024
View Post
  • Public Cloud

PyTorch/XLA 2.5: vLLM support and an improved developer experience

  • October 31, 2024

Stay Connected!
LATEST
  • 1
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 2
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • 5
    TD Synnex named as UK distributor for Cohesity
    • May 28, 2025
  • Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • 7
    Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more
    • May 28, 2025
  • 8
    Pulsant targets partner diversity with new IaaS solution
    • May 23, 2025
  • 9
    Growing AI workloads are causing hybrid cloud headaches
    • May 23, 2025
  • Gemma 3n 10
    Announcing Gemma 3n preview: powerful, efficient, mobile-first AI
    • May 22, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • Understand how Windows Server 2025 PAYG licensing works
    • May 20, 2025
  • By the numbers: How upskilling fills the IT skills gap
    • May 21, 2025
  • 3
    Cloud adoption isn’t all it’s cut out to be as enterprises report growing dissatisfaction
    • May 15, 2025
  • 4
    Hybrid cloud is complicated – Red Hat’s new AI assistant wants to solve that
    • May 20, 2025
  • 5
    Google is getting serious on cloud sovereignty
    • May 22, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.