Engineering Archives - Page 27 of 82 - aster.cloud

7 min

Benchmarking Your Dataflow Jobs For Performance, Cost And Capacity Planning

Calling all Dataflow developers, operators and users…So you developed your Dataflow job, and you’re now wondering how exactly will it perform in the wild, in particular: How many workers does it need to handle your peak load and is there sufficient capacity (e.g. CPU quota)? What is your pipeline’s total cost of ownership (TCO), and is there room to optimize performance/cost ratio? Will the pipeline meet your expected service-level objectives (SLOs) e.g. daily volume, event throughput and/or end-to-end latency? To answer all these questions, you need to performance test your pipeline with real data to measure things like throughput and…

4 min

Meet Optimus, Gojek’s Open-Source Cloud Data Transformation Tool

Editor’s note: Earlier this year, we heard from Gojek, the on-demand services platform, about the open-source data ingestion tool it developed for use with data warehouses like BigQuery. Today, Gojek VP of Engineering Ravi Suhag is back to discuss the open-source data transformation tool it is building. In a recent post, we introduced Firehose, an open source solution by Gojek for ingesting data to cloud data warehouses like Cloud Storage and BigQuery. Today, we take a look at another project within the data transformation and data processing flow. As Indonesia’s largest hyperlocal on-demand services platform, Gojek has diverse data needs…

2 min

Container Analysis Support For Maven And Go Automatic Scanning Of Containers In Public Preview

Java and Go vulnerability scanning support Google Cloud’s Container Scanning API now automatically scans Maven and Go packages for vulnerabilities. With the Container Scanning API enabled, any containers including Java (in Maven repositories) and Go language packages that are uploaded to an Artifact Registry repository will be scanned for vulnerabilities. This capability builds on existing Linux OS based vulnerability detection and provides customers with deeper insight into their applications. This feature is in Public Preview which makes it available to all Google Cloud customers. Get started with Artifact Registry via the instructions for Go or the instructions for Java. How…

3 min

Engineering

How LG CNS Is Creating Future AI Leaders With Immersive Machine Learning Training

As a Korean technology leader specializing in digital transformation (DX), LG CNS partners with customers to help them achieve digital growth across fields including cloud, artificial intelligence (AI), big data, smart factory, and smart logistics. Towards the end of 2020, LG CNS was investigating how to drive digital transformation and growth within their own ranks to demonstrate and extend their core competitive advantage as a consulting service and system integration service provider. By enhancing their internal teams’ AI capabilities, they would ensure they could provide even more innovation and technical expertise for their customers’ own transformation journeys to maintain their…

7 min

Engineering

Google Cloud Firewall Introduces Network Firewall Policies, IAM-governed Tags And More

The network security that firewalls provide is one of the basic building blocks for a secure cloud infrastructure. We are excited to announce that three new Google Cloud Firewall features are now generally available: Global Network Firewall Policies, Regional Network Firewall Policies, and IAM-governed Tags. With these enhancements, Cloud Firewall can help you more easily achieve a Zero Trust network posture with a fully distributed, cloud-native stateful inspection firewall service. In the new Global and Regional Network Firewall Policy structures, we expanded the policy structure down to the Virtual Private Cloud (VPC) level, and made it easier and more scalable…

5 min

Exporting And Analyzing Billing Data Using BigQuery

TL;DR – The best way to analyze your billing data is to set up the billing data export to BigQuery. We’ve created an interactive tutorial to help you through the process! In this last post, we went over a few different options to analyze your Google Cloud billing data. The most powerful and comprehensive way to work with your billing data is to enable exporting to BigQuery. Let’s take a closer look at what this is and why it’s a good choice! What does it do? First things first, exporting your billing data to BigQuery is exactly what it…

5 min

Building Trust In The Data With Dataplex

Analytics data is growing exponentially and so is the dependence on the data in making critical business and product decisions. In fact, the best decisions are said to be the ones which are backed by data. In data, we trust! But do we trust the data ? As the data volumes have grown – one of the key challenges organizations are facing is how to maintain the data quality in a scalable and consistent way across the organization. While data quality is not a newly found need, the needs used to be contained when the data footprint was small and…

4 min

Engineering

Introducing Custom Organization Policy For GKE To Harden Security

Compliance officers and platform engineering teams often find it challenging to ensure security, manage consistency, and oversee governance across multiple products, environments, and teams. Google Cloud’s Organization Policy Service can help tackle this challenge with a policy-based approach that simplifies policy administration across Google Cloud resources and projects. We’re excited to announce the Preview release of Custom Organization Policy, and to showcase the integration with Google Kubernetes Engine (GKE). Custom organization policy for GKE can improve security and efficiency using guardrails you define tailored to your organization’s needs, and it’s offered to Google Cloud customers at no additional cost. A…

6 min

Enabling Real-Time AI With Streaming Ingestion In Vertex AI

Many machine learning (ML) use cases, like fraud detection, ad targeting, and recommendation engines, require near real-time predictions. The performance of these predictions is heavily dependent on access to the most up-to-date data, with delays of even a few seconds making all the difference. But it’s difficult to set up the infrastructure needed to support high-throughput updates and low-latency retrieval of data. Starting this month, Vertex AI Matching Engine and Feature Store will support real-time Streaming Ingestion as Preview features. With Streaming Ingestion for Matching Engine, a fully managed vector database for vector similarity search, items in an index are…

Input your search keywords and press Enter.