When you build highly resilient data analytics pipelines, difficult trade-offs are common. Increasing reliability can come at a cost, but then pursuing low cost can mean more hands-on, operational management. The Cloud Pub/Sub team wants to give you the flexibility to optimize for cost, reliability, and level of management given your workload. So, we’re introducing an additional messaging option that balances cost with high availability. This new Pub/Sub Lite offering is called regional topics.
Using two zones to protect you from zonal outages
Pub/Sub Lite is a high-volume messaging service with ultra-low cost. Previously, to achieve the low cost, all Lite topics were zonal and stored messages in a single zone. Now Pub/Sub Lite is offering you regional topics. Unlike zonal topics, regional topics are asynchronously replicated across two zones instead of one. Replication allows regional topics to achieve high availability with a monthly uptime percentage SLO of >=99.95%, greater than that of zonal topics at >=99.5%.
From our partners:
Balancing cost and availability
If achieving very low cost is a priority, but protecting yourself from zonal outages is a requirement, regional Lite topics are for you. Regional Lite topics strike a balance between zonal Lite topics and the more robust Pub/Sub (which is replicated across three zones) in terms of availability and pricing.
Regional topics are designed to protect your data from server and disk failures. When you publish a message to a regional topic, the service writes the message to disk in the primary zone. When this succeeds, you immediately receive an acknowledgement. In parallel, the same message is copied to the secondary zone. The replication is asynchronous because the system does not wait for the copy to verify that replication succeeded before responding to the client. In the case of individual copy failures, the data on disk is regularly reconciled between the primary and secondary copies.
Additional durability is not the only benefit of regional topics. Partial or complete zonal failures can be caused by machine, program, or operator errors, software upgrades, or natural disasters. In the rare scenario when a zone becomes temporarily unavailable, regional topics will seamlessly failover to the secondary zone. See documentation for details.
The cost of regional Lite topics falls between that of Pub/Sub and zonal Lite topics. Regional Lite topics can be 3 to 4 times more expensive than zonal Lite topics but as little as one third the price of Pub/Sub. An example workload of 10 MiB/s publish throughput with one subscriber has a monthly cost of $169 for a zonal Lite topic, $608 for a regional Lite topic, and $2,000 for Pub/Sub. That’s a savings of nearly 70% going from Pub/Sub to regional Lite topics! Regional Lite topics also require the use of reservations. See the pricing page for details.
Try regional topics in Pub/Sub Lite today
Creating regional topics is nearly identical to creating zonal topics. Simply select a region instead of a zone for topic location. Give it a try in the Google Cloud Console.
To learn more about choosing between Pub/Sub and Pub/Sub Lite, click here.
By: Joanna Young (Product Manager)
Source: Google Cloud Blog
For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!
Our humans need coffee too! Your support is highly appreciated, thank you!
