AWS re:Invent 2024 - How Netflix benchmarks FMs and LLMs across hardware chipsets (NFX307)

Netflix deploys various foundation models on standard CPUs and specialized accelerated computing chips from providers like NVIDIA, AWS, AMD, and Intel. Optimizing instance selection based on price and performance is crucial for rightsizing workloads, achieving cost efficiencies, and accurately forecasting infrastructure needs. In this session, hear Netflix’s approach to automating FM performance benchmarking using FMBench, an open source tool developed by AWS. Learn how FMBench simplifies deployment of FMs to Amazon EC2 and FMBench's reporting capabilities, which capture key performance and accuracy metrics, enabling data-driven decisions based on latency, throughput, and cost requirements.

Learn more:

Subscribe:

About AWS:
Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

#AWSreInvent #AWSreInvent2024