aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Programming
  • Public Cloud

Analyze Pacemaker Events In Cloud Logging

  • aster.cloud
  • April 5, 2022
  • 3 minute read

Customer’s deploying SAP on Google Cloud often leverage Pacemaker for high availability to support their most critical systems. Let’s take a look at how you can use Cloud Logging to easily conduct root cause analysis of Pacemaker clusters.

When there are multiple Pacemaker clusters running in Google Cloud platform, a central logging place can help to store the Pacemaker logs and offer an easy way to analyze Pacemaker events such as fencing or resource failover.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

The Ops Agent is the primary agent for collecting telemetry from your Compute Engine instances. Combining logging and metrics into a single agent, the Ops Agent uses Fluent Bit for logs, which supports high-throughput logging, and the OpenTelemetry Collector for metrics.

 

Install the Agent

Follow this guide to install the Ops Agent on a single VM via command line or using the Google Cloud Console. To install the agent to multiple VMs, use gcloud or automation tools. Ensure your VM doesn’t have the legacy Cloud Logging Agent or Cloud Monitoring Agent installed on it.

 

Configure the Agent

By default, the Ops Agent’s build-in configuration collects file-based syslog log. Pacemaker resource agents such as SAPHana write logs to system log /var/log/messages in SAP certified OS SUSE and RedHat.

Add below configuration elements to the user configuration file /etc/google-cloud-ops-agent/config.yaml to stream Pacemaker logs to Cloud Logging. The path defined below covers all default log files that Pacemaker writes logs to in SUSE and RedHat.

Pacemaker-log is the receiver ID which defines the logName “projects/[PROJECT_ID]/logs/pacemaker-log” of the log entries streamed to Cloud Logging.

Read More  New Cloud Shell Editor: Get Your First Cloud-native App Running In Minutes

Note: If there are existing configurations defined in the logging section, then only add the bold parts.

logging:
  receivers:
    pacemaker-log:
      type: files
      include_paths: 
      - /var/log/pacemaker.log
      - /var/log/cluster/corosync.log
      - /var/log/pacemaker/pacemaker.log
  service:
    pipelines:
      pacemaker-pipeline:
        receivers: [pacemaker-log]

 

Restart the agent

Restart the agent to apply the user-specified configuration

sudo service google-cloud-ops-agent restart

 

Validate the agent

Validate in logging module log

/var/log/google-cloud-ops-agent/subagents/logging-module.log to ensure the Pacemaker logs are activated, you should see similar entries as below listing Pacemaker logs. Follow the troubleshooting guide for any issues.

[2021/08/19 04:34:26] [ info] [sp] stream processor started
[2021/08/19 04:34:26] [ info] [input:tail:tail.3] inotify_fs_add(): inode=51595418 watch_fd=1 name=/var/log/cluster/corosync.log
[2021/08/19 04:34:26] [ info] [input:tail:tail.0] inotify_fs_add(): inode=16814304 watch_fd=1 name=/var/log/messages
[2021/08/19 04:34:26] [ info] [input:tail:tail.2] inotify_fs_add(): inode=51506 watch_fd=1 name=/var/log/google-cloud-ops-agent/subagents/logging-module.log
[2021/08/19 04:34:26] [ info] [input:tail:tail.3] inotify_fs_add(): inode=51595416 watch_fd=2 name=/var/log/pacemaker/pacemaker.log

 

Validate cloud logging

Use below log filter (replace PROJECT_ID) in Cloud Logging Logs Explorer to validate the Pacemaker logs are being streamed there.

logName="projects/[PROJECT_ID]/logs/pacemaker-log"

Now you can use Cloud Logging Logs Explorer to analyze Pacemaker events.  Below sample log filter can help to filter the critical Pacemaker actions and events. Replace the INSTANCE_ID_NODE1/2 with the actual instance IDs of the two cluster nodes. The filter captures

  • Actions of the cluster nodes, cluster resources such as start, stop or promote
  • Failed resource operations, such as start, stop or promote
  • Fencing actions, reasons (loss of cluster nodes, resource failure etc.) and results
  • Corosync communication errors
  • Cluster membership changes, member joins or leaves
resource.type="gce_instance"
(resource.labels.instance_id="[INSTANCE_ID_NODE1]" OR resource.labels.instance_id="[INSTANCE_ID_NODE2]")
(protoPayload.methodName="v1.compute.instances.reset"
OR jsonPayload.message: ("LogNodeActions"
OR ("remote_op_done" "Operation")
OR ("notice" "LogAction")
OR ("TOTEM" "failed" OR "membership" OR "Restransmit")
OR ("Result of" "operation" NOT "ok" NOT "Cancelled" NOT "probe")
OR ("SAPHana(" "WARNING" OR "ERROR")
OR ("SAPInstance(" OR "stonith-ng:" OR "gcp:stonith:" OR "gcp:alias:" OR "gcp-vpc-move-vip:" OR "fence_gce:" "ERROR" OR "Failed")
)
)

Now Pacemaker logs from all your clusters are stored in Cloud Logging, you can analyze Pacemaker events happening to any of your clusters in one central place. If further support is needed from Google Cloud Customer Care Team, efforts and time are saved to collect and transfer logs to the support agent.

Read More  Skaffold V2 GA: Further Enhancing Developer Productivity

To monitor Pacemaker clusters and receive alerts, read this blog What’s happening in your SAP systems? Find out with Pacemaker Alerts.

 

 

By Cherry Legler, Senior Technical Solution Engineer
Source Google Cloud


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Cloud Logging
  • Google Cloud
  • OpenTelemetry Collector
  • Pacemaker
  • SAP
You May Also Like
View Post
  • Computing
  • Public Cloud
  • Technology

United States Army Enterprise Cloud Management Agency Expands its Oracle Defense Cloud Services

  • April 15, 2025
DeepSeek R1 is now available on Azure AI Foundry and GitHub
View Post
  • Public Cloud
  • Technology

DeepSeek R1 is now available on Azure AI Foundry and GitHub

  • February 2, 2025
Cloud platforms among the clouds
View Post
  • Computing
  • Learning
  • Public Cloud

Best Cloud Platforms Offering Free Trials for Cloud Mastery

  • December 23, 2024
Vehicle Manufacturing
View Post
  • Hybrid Cloud
  • Public Cloud

Toyota shifts into overdrive: Developing an AI platform for enhanced manufacturing efficiency

  • December 10, 2024
IBM and AWS
View Post
  • Public Cloud

IBM and AWS Accelerate Partnership to Scale Responsible Generative AI

  • December 2, 2024
COP29 AI and Climate Change
View Post
  • Public Cloud
  • Technology

How Cloud And AI Are Bringing Scale To Corporate Climate Mitigation And Adaptation

  • November 18, 2024
Cloud Workstations
View Post
  • Public Cloud

FEDRAMP High Development in the Cloud: Code with Cloud Workstations

  • November 8, 2024
View Post
  • Public Cloud

PyTorch/XLA 2.5: vLLM support and an improved developer experience

  • October 31, 2024

Stay Connected!
LATEST
  • 1
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 2
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • 5
    TD Synnex named as UK distributor for Cohesity
    • May 28, 2025
  • Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • 7
    Broadcom’s ‘harsh’ VMware contracts are costing customers up to 1,500% more
    • May 28, 2025
  • 8
    Pulsant targets partner diversity with new IaaS solution
    • May 23, 2025
  • 9
    Growing AI workloads are causing hybrid cloud headaches
    • May 23, 2025
  • Gemma 3n 10
    Announcing Gemma 3n preview: powerful, efficient, mobile-first AI
    • May 22, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • Understand how Windows Server 2025 PAYG licensing works
    • May 20, 2025
  • By the numbers: How upskilling fills the IT skills gap
    • May 21, 2025
  • 3
    Cloud adoption isn’t all it’s cut out to be as enterprises report growing dissatisfaction
    • May 15, 2025
  • 4
    Hybrid cloud is complicated – Red Hat’s new AI assistant wants to solve that
    • May 20, 2025
  • 5
    Google is getting serious on cloud sovereignty
    • May 22, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.