aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Engineering

Introducing Datastream For BigQuery

  • aster.cloud
  • October 9, 2022
  • 3 minute read

In today’s competitive environment, organizations need to quickly and easily make decisions based on real-time data. That’s why we’re announcing Datastream for BigQuery, now available in preview, featuring seamless replication from operational database sources such as AlloyDB for PostgreSQL, PostgreSQL, MySQL, and Oracle, directly into BigQuery, Google Cloud’s serverless data warehouse. Datastream for BigQuery is Google’s next big step towards realizing our vision for the unified data cloud, combining databases, analytics, and machine learning into a single platform that offers the scale, speed, security, and simplicity that modern businesses need. With a serverless, auto-scaling architecture, Datastream allows you to easily set up an ELT (Extract, Load, Transform) pipeline for low-latency data replication enabling real-time insights.

Consider the case of a large supermarket chain, with 100’s of stores spread out across the region. Each individual store runs its own local point-of-sale and stock management systems, collecting data throughout the day about transactions and stock-levels in the store. To provide visibility and help streamline the chain’s daily operations, the IT department set up a nightly batch process to collect and consolidate all of the data from the stores into a central data warehouse, so that reports on the stores’ performance could be ready for review in the morning. Maintaining this process took time and resources from the data engineering team, and as the chain grows and more data needs to be processed, this process ended up taking so long that reports would only be ready late into the day. Organizations like this are looking for a modern solution that allows effortless replication of operational data to their data warehouse, enabling real-time decision making; Datastream for BigQuery is that solution.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Read More  Intel Collaborates With Google Cloud To Accelerate Newest VMs

Datastream accelerates data-driven decision making in BigQuery

Developed in close partnership with Google Cloud’s BigQuery team, Datastream for BigQuery delivers a unique, truly seamless and easy-to-use experience that enables real-time insights in BigQuery with just a few steps.

Using BigQuery’s newly developed Change Data Capture (CDC) and Storage Write API’s UPSERT functionality, Datastream efficiently replicates updates directly from source systems into BigQuery tables in real-time. You no longer have to waste valuable resources building and managing complex data pipelines, self-managed staging tables, tricky DML merge logic, or manual conversion from database-specific data types into BigQuery data types. Just configure your source database, connection type, and destination in BigQuery and you’re all set. Datastream for BigQuery will backfill historical data and continuously replicate new changes as they happen. And as database schemas shift, Datastream seamlessly handles schema changes and automatically adds new tables and columns to BigQuery.

New volume-based tiered pricing

We are also excited to announce the launch of volume-based tiered pricing that makes it more affordable for customers moving larger volumes of data. Volume-based tiered pricing will be applied automatically based on actual usage to unlock the power of Datastream.

Klook, a leading travel and leisure e-commerce platform for experiences and services, processes vast amounts of data across a range of applications and databases. Using BigQuery, Klook’s data team produces daily reports and analysis for their management team to help drive better business decisions. “Dealing with complex data environments and ingesting data from different sources into our data warehouse is very challenging”, says Stacy Zhu, Senior Manager for Data at Klook. “Prior to adopting Datastream, we had a team of data engineers dedicated to the task of ingesting data into BigQuery, and we spent a lot of time and effort making sure that the data was accurate. With Datastream, our data analysts can have accurate data readily available to them in BigQuery with a simple click. We enjoy Datastream’s ease-of-use, and its performance helps us achieve large scale ELT data processing.”

Read More  Meet The People Of Google Cloud: Grace Mollison, Solutions Architect And Professional Problem Solver

 

Achievers, an award-winning employee engagement software and platform, is another customer who recently adopted Datastream. “Achievers had been heavily using Google Cloud VMs (GCE), and Google Kubernetes Engine (GKE)” says Daljeet Saini, Lead Data Architect at Achievers. “With the help of Datastream, Achievers will be streaming our data into BigQuery and enabling our analysts and data scientists to start using BigQuery for smart analytics, helping us take the data warehouse to the next level.”

Start using Datastream today

You can get started today with Datastream, available to all customers in all Google Cloud regions. For more information on Datastream for BigQuery, please check out the product page.

 

 

By: Andi Gutmans (GM and VP of Engineering for Databases, Google Cloud) and Etai Margolin (Product Manager)
Source: Google Cloud Blog


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • BigQuery;
  • Data Analytics
  • Datastream
  • Google Cloud
You May Also Like
View Post
  • Engineering
  • Technology

Apple supercharges its tools and technologies for developers to foster creativity, innovation, and design

  • June 9, 2025
View Post
  • Engineering

Just make it scale: An Aurora DSQL story

  • May 29, 2025
View Post
  • Engineering
  • Technology

Guide: Our top four AI Hypercomputer use cases, reference architectures and tutorials

  • March 9, 2025
View Post
  • Computing
  • Engineering

Why a decades old architecture decision is impeding the power of AI computing

  • February 19, 2025
View Post
  • Engineering
  • Software Engineering

This Month in Julia World

  • January 17, 2025
View Post
  • Engineering
  • Software Engineering

Google Summer of Code 2025 is here!

  • January 17, 2025
View Post
  • Data
  • Engineering

Hiding in Plain Site: Attackers Sneaking Malware into Images on Websites

  • January 16, 2025
View Post
  • Computing
  • Design
  • Engineering
  • Technology

Here’s why it’s important to build long-term cryptographic resilience

  • December 24, 2024

Stay Connected!
LATEST
  • 1
    Building secure, scalable AI in the cloud with Microsoft Azure
    • July 5, 2025
  • 2
    Turns out OpenAI is the customer behind Oracle’s mysterious $30 billion cloud deal
    • July 3, 2025
  • aster-cloud-erp-bill_of_materials_2 3
    What is an SBOM (software bill of materials)?
    • July 2, 2025
  • aster-cloud-sms-pexels-tim-samuel-6697306 4
    Send SMS texts with Amazon’s SNS simple notification service
    • July 1, 2025
  • Camping 5
    The Summer Adventures : Camping Essentials
    • June 27, 2025
  • aster-cloud-website-pexels-goumbik-574069 6
    Host a static website on AWS with Amazon S3 and Route 53
    • June 27, 2025
  • 7
    A looming hyperscaler exodus? UK IT leaders are thinking of ditching US cloud providers – here’s why
    • June 26, 2025
  • Prioritize security from the edge to the cloud
    • June 25, 2025
  • 6 edge monitoring best practices in the cloud
    • June 25, 2025
  • Genome 10
    AlphaGenome: AI for better understanding the genome
    • June 25, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • 1
    There’s a ‘cloud reset’ underway, and VMware Cloud Foundation 9.0 is a chance for Broadcom to pounce on it
    • June 17, 2025
  • 2
    ‘This was a very purposeful strategy’: Pure Storage unveils Enterprise Data Cloud in bid to unify data storage, management
    • June 18, 2025
  • 3
    Pure Accelerate 2025: All the news and updates live from Las Vegas
    • June 18, 2025
  • Oracle adds xAI Grok models to OCI
    • June 17, 2025
  • What is cloud bursting?
    • June 18, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.