BigQuery Archives - aster.cloud

6 min

From Receipts To Riches: Save Money W/ Google Cloud & Supermarket Bills – Part 1

In today’s world, every penny counts, and saving money on supermarket spending is no exception. Have you ever wondered: This is the first part of a two-part blog series that demonstrates how simple it is to combine managed services on Google Cloud to create a complete application that digitizes your grocery receipts and analyzes your spending patterns using Google Cloud services. This architecture is useful for any number of use cases where you want to apply the power of document AI to fully event-driven processing pipelines. In this first part of the blog series, we will discuss how to extract…

5 min

Sentiment Analysis With BigQuery ML

Introduction We recently announced BigQuery support for sparse features which help users to store and process the sparse features efficiently while working with them. That functionality enables users to represent sparse tensors and train machine learning models directly in the BigQuery environment. Being able to represent sparse tensors is a useful feature because sparse tensors are used extensively in encoding schemes like TF-IDF as part of data pre-processing in NLP applications and for pre-processing images with a lot of dark pixels in computer vision applications. There are numerous applications of sparse features such as text generation and sentiment analysis. In…

5 min

Shorten The Path To Insights With Aiven For Apache Kafka And Google BigQuery

Every company aims to be data driven, but bringing accurate data in front of the right stakeholders in a timely manner can be quite complex. The challenge arises even more when the source data resides in different technologies, with various access interfaces and data formats. This is where the combination of Aiven for Apache Kafka® and Google BigQuery excels, by providing the ability to source the data from a wide ecosystem of tools and, in streaming mode, push it to BigQuery where datasets can be organized, manipulated and queried. From data sources to Apache Kafka with Kafka Connect Aiven, together…

7 min

Building Streaming Data Pipelines On Google Cloud

Many customers build streaming data pipelines to ingest, process and then store data for later analysis. We’ll focus on a common pipeline design shown below. It consists of three steps: For the processing component, we’ll review three alternatives, ranging from basic to advanced: a BigQuery subscription, a Cloud Run service, and a Dataflow pipeline. Example use cases Before we dive deeper into the implementation details, let’s look at a few example use cases of streaming data pipelines: Ingesting data with Pub/Sub Let’s start at the beginning. You have one or multiple data sources that publish messages to a Pub/Sub topic.…

7 min

Built With BigQuery: Aible’s Serverless Journey To Challenge The Cost Vs. Performance Paradigm

Aible is the leader in generating business impact from AI in less than 30 days by helping teams go from raw data to business value with solutions for customer acquisition, churn prevention, demand prediction, preventative maintenance, and more. These solutions help IT and data teams identify valuable data through automated data validation, enabling collaborative open-world exploration of data, and deliver AI recommendations in enterprise applications to help teams achieve business goals while considering unique business circumstances such as marketing budgets and changing market conditions. For example, if a sales optimization model would require a 10% increase in sales resources for…

10 min

Built With BigQuery: How Oden Provides Actionable Recommendations With Network Resiliency To Optimize Manufacturing Processes

Background The Oden Technologies solution is an analytics layer for manufacturers that combines and analyzes all process information from machines and production systems to give real-time visibility to the biggest causes of inefficiency and recommendations to address them. Oden empowers front-line plant teams to make effective decisions, such as prioritizing resources more effectively, solving issues faster, and realizing optimal behavior. Use cases: Challenges and problems solved Manufacturing plants have limited resources and would like to use them optimally by eliminating any inefficiencies and making recommendations and providing data points as a key input for decision making. These data points are…

8 min

Computing

How To Use Advance Feature Engineering To Preprocess Data In BigQuery ML

Preprocessing and transforming raw data into features is a critical but time consuming step in the ML process. This is especially true when a data scientist or data engineer has to move data across different platforms to do MLOps. In this blogpost, we describe how we streamline this process by adding two feature engineering capabilities in BigQuery ML. Our previous blog outlines the data to AI journey with BigQuery ML, highlighting two powerful features that simplify MLOps – data preprocessing functions for feature engineering and the ability to export BigQuery ML TRANSFORM statement as part of the model artifact. In…

SQL

9 min

Demystifying BigQuery BI Engine

BigQuery BI Engine is a fast, in-memory analysis system for BigQuery currently processing over 2 billion queries per month and growing. BigQuery has its roots in Google’s Dremel system and is a data warehouse built with scalability as a goal. On the other hand BI Engine was envisioned with data analysts in mind and focuses on providing value on Gigabyte to sub-Terabyte datasets, with minimal tuning, for real time analytics and BI purposes. Using BI Engine is simple – create a memory reservation on the project that runs BigQuery queries, and it will cache data and use the optimizations. This…

4 min

Tools

BigQuery Authorized Views Permissions Via Terraform, Avoiding The Chicken & Egg Problem

Enterprises that use Terraform for spinning up their Infrastructure, including the instantiation of Google BigQuery, can run into a chicken & egg problem if using the IAM access permissions resource blocks for both their Datasets and Authorized Views. This problem can cause BigQuery operational issues across an organization, creating an unpleasant experience for the end-user due to the momentary loss of access to the data. End users without access to “private data” are likely to rely on the Authorized views to a great extent. This blog post shows how to avoid running into the problem and provides a step-by-step guide…

8 min

You Get What You Pay For: Principles For Designing A Chargeback Process

Principles for designing a chargeback process As large organizations increase their cloud footprint, it becomes critical to ensure costs are being managed effectively. A good understanding of the cost of running each workload, and the value that workload brings to your business, allows organizations to have confidence in the efficiency of their cloud consumption — which is why we see many customers who are successful in driving financial accountability and cost efficiency embrace the Cloud FinOps framework. One crucial capability of Cloud FinOps is chargeback, which is the process of mapping cloud consumption to internal consumers within your organization and…

Input your search keywords and press Enter.