aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Programming
  • Software Engineering

Announcing The Release Of VMware Tanzu RabbitMQ 1.2

  • aster.cloud
  • November 30, 2021
  • 5 minute read

The Tanzu RabbitMQ team is excited to announce the general availability of VMware Tanzu RabbitMQ 1.2.

This version contains RabbitMQ 3.9, a milestone core broker release that introduces Streams, a new data structure allowing for replay and higher throughput.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

In addition to Streams, Tanzu RabbitMQ 1.2 has some exciting new capabilities and improvements:

  • Warm standby replication (active/passive) for disaster recovery (see full details below)

  • Support for HashiCorp Vault as a source for cluster secrets that can be used instead of Kubernetes secrets

  • Enhanced support for RabbitMQ clusters on Red Hat OpenShift

  • Improvements for monitoring queue health

  • A number of bug fixes and stability improvements (see the release notes)

Enterprise-grade disaster recovery for RabbitMQ

Tanzu RabbitMQ has a new mechanism for disaster recovery that allows customers to easily configure standby replication. This capability provides the following:

  • Fast, data-safe message replication – Uses the latest in RabbitMQ protocols and best practices

  • Automatic downstream cluster protection – Automatically prunes messages that have been processed on the active upstream site

  • Easy setup – No need to calculate or assume message throughput rates to configure message expiration/TTL (time to live)

  • Provides faster failover – Downstream applications will only see messages that haven’t been processed on the primary site, reducing the time to recover

Previously, users had to manually configure the federation plug-in, as well as set appropriate message TTLs using an estimate of publishing and consumption speed. This introduced risk into a disaster recovery setup as publishing and consumption rates cannot always be guaranteed in distributed systems.

The new solution does not make any assumptions about publishing and consumption speed, queue depth and memory size; instead it avoids storing messages in the passive cluster queues altogether. It replicates not just the messages, but also information about whether the message has been processed. This tells the passive cluster which messages don’t need to be retained anymore.

Read More  Google Cloud Next 2019 | Multi-Cloud Kubernetes: Formulas for Success

How does it work?

Here is a breakdown of how the new data replication works:

  1. The user defines which vHosts will be backed up and which queues will be backed up using a policy.

  2. Every message written to a Quorum Queue matching the replication policy on the active cluster is logged into the local replication log. There is a replication log for each vHost with a replication policy.

  3. The active cluster logs some metrics in a defined short interval. These will be used by the passive cluster to truncate consumed messages.

  4. Clusters that are configured as passive followers establish links to the active cluster and register as consumers for the replication log. These passive clusters get log entries pushed to them continuously as new entries are logged. A passive cluster can be linked at any given time, and there can be multiple passive clusters per active cluster.

  5. The passive cluster logs these entries into equivalent local replication logs.

  6. When the passive cluster is promoted to become active using an API call, it will read from the unconsumed messages from the local replication log and will write them into the local queues.

 

As a result of this design, the passive cluster is not only protected from queue overflow or losing data by premature message deletion due to a poorly estimated TTL, it also enjoys near real-time replication (only limited by network speed) and quick recovery time objective (RTO) since minimal unconsumed messages need to be enqueued for processing.

This capability is available as part of Tanzu RabbitMQ. The easiest implementation path is to deploy Tanzu RabbitMQ on Kubernetes, where a new operator automates the configuration of replication on both clusters (active and passive), making setup fast and simple.

An application team can now be given not just self-service creation of their RabbitMQ cluster but also the ability to self-create a disaster recovery standby cluster with replication. A task once done by Ops SMEs, often taking weeks to months, can now be done by each application team in a matter of a few minutes.

Read More  Equinix Expands Collaboration With VMware To Help Enterprises Accelerate Digital Transformation

Integration with HashiCorp Vault

We understand that Tanzu RabbitMQ is one of many different workloads that our users want to manage in their Kubernetes environment. We know that our users typically have a central process for managing secrets, and we would like to help them use the DevSecOps toolchain of their choice. We chose to start with HashiCorp Vault because it is one of the most popular secret stores for microservices and cloud native applications. Vault has several ways to integrate with Kubernetes workloads, and in this version we have integrated our cluster operator with the Vault agent in order to allow users to manage RabbitMQ default user credentials as well as TLS certificates using Vault.

The flow is quite simple and works with a user’s own process to store the secrets in Vault:

  • Install the Vault sidecar injector on the Kubernetes cluster

  • Configure the Vault agent context using the cluster operator API YAML

    • Specify Vault as the secret back end to be used

    • Specify the Vault role for this cluster

    • Optionally, the user can also specify the annotations for the Vault sidecar

  • Once the user applies the YAML, the cluster operator will create a RabbitMQ cluster. Each pod of this cluster will have the annotations that will cause Kubernetes to inject the Vault sidecar container into the pod. This means that next to each RabbitMQ node, there is a Vault agent. The Vault agent can now authenticate with the Vault server using the Kubernetes service account token and expose the Vault-managed secrets to the pod containers as a file system mount.

Read More  6 Unspoken Laws Of Strategic Execution

While RabbitMQ will read the certificates from the file system and doesn’t need any additional steps to use a rotated certificate, rotating the default user password requires an additional API call. An additional sidecar does exactly that, watching the Vault agent file system and rotating the password when it changes.

Here are a few examples:

  • Vault default user

  • Vault TLS

  • Video: How to use HashiCorp Vault as external secret provider in RabbitMQ/cluster-operator

Monitoring queue health

RabbitMQ exposes health metrics through a Prometheus endpoint. There is an existing /metrics path that exposes both general metrics about Erlang VM and also RabbitMQ-specific metrics about queues, connections, etc. By default, this endpoint operates in the aggregated mode, so that only totals can be observed, i.e., the total number of ready messages in all of the queues. It can be switched to a per-object mode (via configuration setting, or by using /metrics/per-object URL), where separate objects can be seen in metric labels, such as queue and exchange names.

However, per-object mode exposes every metric known to the system, which can be unreasonable when there are a lot of objects. That’s why a new /metrics/detailed was introduced, which allows one to collect only those per-object metrics that are of interest to them, or filter them on a per-vhost basis. Collecting only the number of messages and number of consumers per queue is significantly faster than collecting everything, yet provides enough information for meaningful monitoring.

In addition, observability examples are now configured to scrape this new endpoint for a minimal set of per-object metrics, which allow the following two preconfigured Prometheus alerts:

  • Alert when there are non-empty queues without consumers

  • Alert when there are queues where number of messages is steadily growing

 

By YARON PARASOL
Source VMware Tanzu


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • DevSecOps
  • HashiCorp Vault
  • Tanzu
  • Tanzu RabbitMQ
  • VMware
You May Also Like
View Post
  • Software Engineering
  • Technology

Claude 3.7 Sonnet and Claude Code

  • February 25, 2025
View Post
  • Engineering
  • Software Engineering

This Month in Julia World

  • January 17, 2025
View Post
  • Engineering
  • Software Engineering

Google Summer of Code 2025 is here!

  • January 17, 2025
View Post
  • Software Engineering

5 Books Every Beginner Programmer Should Read

  • July 25, 2024
Ruby
View Post
  • Software Engineering

How To Get Started With A Ruby On Rails Project – A Developer’s Guide

  • January 27, 2024
View Post
  • Engineering
  • Software Engineering

5 Ways Platform Engineers Can Help Developers Create Winning APIs

  • January 25, 2024
Clouds
View Post
  • Cloud-Native
  • Platforms
  • Software Engineering

Microsoft Releases Azure Migrate Assessment Tool For .NET Application

  • January 14, 2024
View Post
  • Software Engineering
  • Technology

It’s Time For Developers And Enterprises To Build With Gemini Pro

  • December 21, 2023

Stay Connected!
LATEST
  • 1
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 2
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • 5
    TD Synnex named as UK distributor for Cohesity
    • May 28, 2025
  • Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • 7
    Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more
    • May 28, 2025
  • 8
    Pulsant targets partner diversity with new IaaS solution
    • May 23, 2025
  • 9
    Growing AI workloads are causing hybrid cloud headaches
    • May 23, 2025
  • Gemma 3n 10
    Announcing Gemma 3n preview: powerful, efficient, mobile-first AI
    • May 22, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • Understand how Windows Server 2025 PAYG licensing works
    • May 20, 2025
  • By the numbers: How upskilling fills the IT skills gap
    • May 21, 2025
  • 3
    Cloud adoption isn’t all it’s cut out to be as enterprises report growing dissatisfaction
    • May 15, 2025
  • 4
    Hybrid cloud is complicated – Red Hat’s new AI assistant wants to solve that
    • May 20, 2025
  • 5
    Google is getting serious on cloud sovereignty
    • May 22, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.