Posts in tag

Python

DataEngineeringSoftware EngineeringTechnology

This article is part-II of the series on ‘PyTorch/XLA:Performance Debugging on TPU-VM’. In the previous article we introduced the basic metrics of performance analysis. We used the client side debugging with the PyTorch/XLA profiler to identify how the .equal() operator used inside the Multihead Attention module implementation caused frequent recompilation of the graph causing  the …

Read More
ProgrammingSoftware EngineeringTechTools

In this three part series we explore the performance debugging ecosystem of PyTorch/XLA on Google Cloud TPU VM. TPU VM earlier this year (2021). The TPU VM architecture allows the ML practitioners to work directly on the host where TPU hardware is attached. With the TPU profiler launched earlier this year, debugging your PyTorch training …

Read More
DataProgramming

In this tutorial, you’ll be introduced to 6 handy Python data visualization libraries, and learn how to choose among them. Data visualization is a great way to analyze and present information; Python is one of the most popular data analysis languages. This results in many good Python libraries for data visualizations. There is no single best …

Read More
DataEngineeringPractices

We’re very excited to announce the general availability of the Google Cloud Spanner dialect that enables SQLAlchemy applications to take advantage of Cloud Spanner‘s scale, strong consistency, and up to 99.999% availability. SQLAlchemy is a Python SQL toolkit and Object Relational Mapper, giving application developers the full power and flexibility of SQL. Its object-relational mapper …

Read More
DataEngineeringProgrammingSoftwareTechnology

SlashData’s latest Developer Nation report looks at the popularity of programming languages and trends from the global developer community.  JavaScript is now used by more than 16.4 million developers globally, says a survey of more than 19,000 coders – making it the world’s most popular programming language “by a wide margin”. SlashData’s 21st State of the …

Read More
DataEngineeringPracticesProgrammingTechnology

Techniques on what to clean and how. Before fitting a machine learning or statistical model, we always have to clean the data. No models create meaningful results with messy data. Data cleaning or cleansing is the process of detecting and correcting (or removing) corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, …

Read More
Software EngineeringSolutionsTechnology

Editor’s note: Over the past several weeks, we’ve posted a series of blog posts focusing on best practices for writing Google Cloud Functions based on common questions or misconceptions as seen by the Support team.  We refer to these as “anti-patterns” and offer you ways to avoid them.  This article is the fifth post in …

Read More
code
EngineeringSoftware EngineeringTools

Cloud Storage is a common choice for Vertex AI and AI Platform users to store their training data, models, checkpoints and logs. Now, with Cloud Storage FUSE, training jobs on both platforms can access their data on Cloud Storage as files in the local file system. This post introduces the Cloud Storage FUSE for Vertex …

Read More
ProgrammingSoftwareTechnologyTools

Use setuptools to deliver Python code to users. You’ve spent weeks perfecting your code. You’ve tested it and sent it to some close developer friends for quality assurance. You’ve posted all the source code on your personal Git server, and you’ve received helpful bug reports from a few brave early adopters. And now you’re ready to …

Read More
ProgrammingPublic Cloud

For many developers that come to Dataflow, Google Cloud’s fully managed data processing service, the first decision they have to make is which programming language to use. Dataflow developers use the open-source Apache Beam SDK to author their pipelines, and have several choices for language to use: Java, Python, Go, SQL, Scala, and Kotlin. In …

Read More