aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Computing

Microsoft Launches Latest Azure Virtual Machines Optimized For AI Supercomputing, The ND H200 V5 Series

  • aster.cloud
  • October 8, 2024
  • 2 minute read

The need for scalable and high-performance infrastructure continues to grow exponentially as the AI landscape advances. Our customers rely on Azure AI infrastructure to develop innovative AI-driven solutions, which is why we are delivering new cloud-based AI-supercomputing clusters built with Azure ND H200 v5 series virtual machines (VMs) today. These VMs are now generally available and have been tailored to handle the growing complexity of advanced AI workloads, from foundational model training to generative inferencing. The scale, efficiency and enhanced performance of our ND H200 v5 VMs are already driving adoption from customers and Microsoft AI services such as Azure Machine Learning and Azure OpenAI Service.

“We’re excited to adopt Azure’s new H200 VMs. We’ve seen that H200 offers improved performance with minimal porting effort, we are looking forward to using these VMs to accelerate our research, improve the ChatGPT experience, and further our mission.” —Trevor Cai, head of infrastructure, OpenAI.

The Azure ND H200 v5 VMs are architected with Microsoft’s systems approach to enhance efficiency and performance, and feature eight NVIDIA H200 Tensor Core GPUs. Specifically, they address the gap due to GPUs growing in raw computational capability at a much faster rate than the attached memory and memory bandwidth. The Azure ND H200 v5 series VMs deliver a 76% increase in High Bandwidth Memory (HBM) to 141GB and a 43% increase in HBM Bandwidth to 4.8 TB/s over the previous generation of Azure ND H100 v5 VMs. This increase in HBM bandwidth enables GPUs to access model parameters faster, helping reduce overall application latency, which is a critical metric for real-time applications such as interactive agents. The ND H200 V5 VMs can also accommodate more complex Large Language Models (LLMs) within the memory of a single VM, improving performance by helping users avoid the overhead of running distributed jobs over multiple VMs. 


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Read More  6 Finops Best Practices To Reduce Cloud Costs

The design of our H200 supercomputing clusters also enables more efficient management of GPU memory for model weights, key-value cache, and batch sizes, all of which directly impact throughput, latency and cost-efficiency in LLM-based generative AI inference workloads. With its larger HBM capacity, the ND H200 v5 VM can support higher batch sizes, driving better GPU utilization and throughput compared to ND H100 v5 series for inference workloads on both small language models (SLMs) and LLMs. In early tests, we observed up to 35% throughput increase with ND H200 v5 VMs compared to the ND H100 v5 series for inference workloads running the LLAMA 3.1 405B model (with world size 8, input length 128, output length 8, and maximum batch sizes – 32 for H100 and 96 for H200). For more details on Azure’s high performance computing benchmarks, please read more here or visit our AI Benchmarking Guide on the Azure GitHub repository for more details. 

The ND H200 v5 VMs come pre-integrated with Azure Batch, Azure Kubernetes Service, Azure OpenAI Service and Azure Machine Learning to help businesses get started right away. Please visit here for more detailed technical documentation of the new Azure ND H200 v5 VMs. 

By: Nitin Nagarkatte, Principal Product Manager, Azure HPC+AI
Originally published at: Microsoft Azure Blog

Source: zedreviews.com


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • Azure
  • Cloud Computing
  • Compute
  • Microsoft
  • Supercomputing
  • Virtual Machine
You May Also Like
View Post
  • Computing
  • Multi-Cloud
  • Technology

How to create an AWS free tier account

  • July 10, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

How to configure multiple AWS CLI authentication credentials

  • July 10, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

What is database as a service (DBaaS)?

  • July 7, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

The cloud’s role in PQC migration

  • July 7, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Hybrid cloud has hit the mainstream – but firms are still confused about costs

  • July 7, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Turns out OpenAI is the customer behind Oracle’s mysterious $30 billion cloud deal

  • July 3, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

A looming hyperscaler exodus? UK IT leaders are thinking of ditching US cloud providers – here’s why

  • June 26, 2025
View Post
  • Computing
  • Multi-Cloud
  • Technology

Prioritize security from the edge to the cloud

  • June 25, 2025

Stay Connected!
LATEST
  • How to create an AWS free tier account
    • July 10, 2025
  • How to configure multiple AWS CLI authentication credentials
    • July 10, 2025
  • 3
    Formula E accelerates its work with Google Cloud Storage and Google Workspace
    • July 9, 2025
  • What is database as a service (DBaaS)?
    • July 7, 2025
  • The cloud’s role in PQC migration
    • July 7, 2025
  • 6
    Hybrid cloud has hit the mainstream – but firms are still confused about costs
    • July 7, 2025
  • 7
    Building secure, scalable AI in the cloud with Microsoft Azure
    • July 5, 2025
  • 8
    Turns out OpenAI is the customer behind Oracle’s mysterious $30 billion cloud deal
    • July 3, 2025
  • aster-cloud-erp-bill_of_materials_2 9
    What is an SBOM (software bill of materials)?
    • July 2, 2025
  • aster-cloud-sms-pexels-tim-samuel-6697306 10
    Send SMS texts with Amazon’s SNS simple notification service
    • July 1, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • 1
    A looming hyperscaler exodus? UK IT leaders are thinking of ditching US cloud providers – here’s why
    • June 26, 2025
  • Genome 2
    AlphaGenome: AI for better understanding the genome
    • June 25, 2025
  • aster-cloud-website-pexels-goumbik-574069 3
    Host a static website on AWS with Amazon S3 and Route 53
    • June 27, 2025
  • Camping 4
    The Summer Adventures : Camping Essentials
    • June 27, 2025
  • 6 edge monitoring best practices in the cloud
    • June 25, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.