aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Design
  • Engineering
  • Work & Jobs

Workflows Patterns And Best Practices – Part 1

  • aster.cloud
  • December 21, 2022
  • 4 minute read
For the last couple of years, we’ve been using Workflows, Google Cloud’s service orchestrator, to bring order to our serverless microservices architectures. As we used and gained more experience with Workflows and service orchestration, we shared what he had learned in conference talks, blog posts, samples, and tutorials. Along the way, some common patterns and best practices emerged.
To help you take better advantage of Workflows and service orchestration on Google Cloud, we’ve summarized these proven patterns and best practices in a three-part series of blog posts.Let’s get started with Part 1!

Make a conscious choice on the communication style upfront

Choosing a communication style is more of a task than a pattern, but it is an important one to complete before even considering service orchestration.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

When you have multiple services, you need to decide how these services will communicate. The options are:

  • Directservice-to-service communication
  • Indirect event-driven communication (also known as choreography)
  • A central orchestrator (e.g. Workflows) directing the communication

There’s no right or wrong, only pros and cons. Direct service-to-service communication is easy to implement but creates tight coupling between services. Events enable loosely coupled services at the expense of harder monitoring and debugging when something goes wrong. An orchestrator, while less flexible, brings order to communication without the tight coupling of direct service-to-service communication and the chaos of events in choreographed architectures.

In our Orchestrating the Pic-a-Daily serverless app with Workflows post, we explained how we transformed an event-driven application into an orchestrated application and the benefits of doing so. In Choosing the right orchestrator in Google Cloud, we talked about which service is best suited for different orchestration needs (scheduled, service, and data).

Read More  Underestimating the Path & Pitfalls to AI Progress

As you design your architecture, make a conscious choice on the communication style with pros and cons in mind, and if you choose to use orchestration, be sure to use the right orchestrator for the task.

Keep these tips and tricks for Workflows in mind

Once you decide to use Workflows for service orchestration, you’ll realize that Workflows has its own strengths and idiosyncrasies. Here are some general tips and tricks that we found useful as we used Workflows:

  • Avoid hard-coding URLs for more portable workflows across environments.
  • Use substeps to collect a common series of steps in one logical unit.
  • Wrap string expressionsto avoid parsing problems.
  • Replace logic-less services with declarative API calls to avoid boilerplate code.
  • Store what you need, free what you can to keep memory consumption under control.
  • Use subworkflows and call external workflows to increase reuse.

Check our post on Workflows tips and tricks for a more detailed explanation of these points.

Consider event-driven orchestration

The choice on the communication style is not all or nothing. You can and should combine different styles when it makes sense. For example, there is a common pattern where services that are closely related are managed by an orchestrator like Workflows but that orchestration is triggered by an event from a service like Eventarc. Similarly, we see architectures where the end of an orchestration is a Pub/Sub message to some other orchestration or service.

In our Introducing Eventarc triggers for Workflows post, we showed how easy it is to route events to Workflows using Eventarc. In the Build an event-driven orchestration with Eventarc and Workflows video and its associated codelab and sample, we showed how to design an image processing pipeline where the services are managed by Workflows but the orchestration is triggered in a loosely coupled way by a Cloud Storage event via Eventarc:

Read More  Bootstrap Your Startup With The Google Cloud Technical Guides For Startups : A Look Into The Start Series

Mix communication styles to get the best of both worlds: use orchestration when you need tight coupling between services but loose coupling with other orchestrations via events.

Use connectors when you can

Workflows has a rich set of connectors to call other Google Cloud services. They handle the formatting of requests for you, providing methods and arguments so that you don’t need to get deep into the gory details of a Google Cloud API. More importantly, connectors enable a workflow to transparently wait for long-running Cloud API calls. This saves you from the tedious work of iterating and waiting for calls to complete; connectors take care of this for you!

In our Introducing new connectors for Workflows post, we showed you how Compute Engine connector simplified creating and stopping a virtual machine.

Whenever you want to call a Google Cloud API from Workflows, check to see if there’s a connector for it. You’ll be glad that you did and you can always request a new connector here, if there’s no connector.

Parallelize when you can

When we talk about Workflows, we often talk about steps executed one after another sequentially. While Workflows is fast enough to run steps sequentially with no noticeable delay, not all steps need to run sequentially. Independent steps can actually run in parallel, and in some cases this can provide a significant speed up for workflow executions.

In the Introducing Parallel Steps for Workflows post and its associated video, we showed how running BigQuery jobs from Workflows in parallel can speed up the workflow execution by five times. The more independent steps you have, the more you can run those steps in parallel and the faster your workflow execution will be, especially with long-running tasks like BigQuery jobs.

Read More  IBM Report: Identity Comes Under Attack, Straining Enterprises' Recovery Time From Breaches

 

Try to keep your steps independent and make sure to take advantage of parallel steps when you can.


This initial list of patterns and tips will help you get started taking better advantage of Workflows. We cover more advanced patterns in part 2 of this series. For questions or feedback, feel free to reach out to us on Twitter @meteatamel and @glaforge.

 

 

By: Guillaume Laforge (Developer Advocate) and Mete Atamel (Developer Advocate)
Source: Google Cloud Blog


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Google Cloud
  • Microservices
  • Tutorials
  • Workflows
You May Also Like
View Post
  • Engineering

Just make it scale: An Aurora DSQL story

  • May 29, 2025
View Post
  • Engineering
  • Technology

Guide: Our top four AI Hypercomputer use cases, reference architectures and tutorials

  • March 9, 2025
View Post
  • Computing
  • Engineering

Why a decades old architecture decision is impeding the power of AI computing

  • February 19, 2025
View Post
  • Engineering
  • Software Engineering

This Month in Julia World

  • January 17, 2025
View Post
  • Engineering
  • Software Engineering

Google Summer of Code 2025 is here!

  • January 17, 2025
View Post
  • Data
  • Engineering

Hiding in Plain Site: Attackers Sneaking Malware into Images on Websites

  • January 16, 2025
View Post
  • Computing
  • Design
  • Engineering
  • Technology

Here’s why it’s important to build long-term cryptographic resilience

  • December 24, 2024
IBM and Ferrari Premium Partner
View Post
  • Data
  • Engineering

IBM Selected as Official Fan Engagement and Data Analytics Partner for Scuderia Ferrari HP

  • November 7, 2024

Stay Connected!
LATEST
  • 1
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 2
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • 5
    TD Synnex named as UK distributor for Cohesity
    • May 28, 2025
  • Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • 7
    Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more
    • May 28, 2025
  • 8
    Pulsant targets partner diversity with new IaaS solution
    • May 23, 2025
  • 9
    Growing AI workloads are causing hybrid cloud headaches
    • May 23, 2025
  • Gemma 3n 10
    Announcing Gemma 3n preview: powerful, efficient, mobile-first AI
    • May 22, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • 1
    Cloud adoption isn’t all it’s cut out to be as enterprises report growing dissatisfaction
    • May 15, 2025
  • 2
    Hybrid cloud is complicated – Red Hat’s new AI assistant wants to solve that
    • May 20, 2025
  • 3
    Google is getting serious on cloud sovereignty
    • May 22, 2025
  • oracle-ibm 4
    Google Cloud and Philips Collaborate to Drive Consumer Marketing Innovation and Transform Digital Asset Management with AI
    • May 20, 2025
  • notta-ai-header 5
    Notta vs Fireflies: Which AI Transcription Tool Deserves Your Attention in 2025?
    • May 16, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.