Strategies for Efficient LLM Deployments in Any Cluster -Angel M De Miguel Meana & Francisco Cabrera

preview_player
Показать описание

Strategies for Efficient LLM Deployments in Any Cluster - Angel M De Miguel Meana, VMware & Francisco Cabrera, Microsoft

Undoubtedly, Large Language Models (LLMs) are the technological advancement of 2023. These models show many capabilities, from chatting like a historical character to converting unstructured data into JSON format. However, their substantial size (GBs), resource demands, and the management complexity present considerable challenges. At the same time, Kubernetes has emerged as the de facto technology for orchestrating workloads, and LLMs are no exception. In this talk, we will explore multiple strategies to reduce the footprint of these models in your cluster, making it possible to move them from the cloud to the edge. We will answer questions like how to select the right model, reduce their size, and optimize resource utilization by running them in a lightweight environment provided by WebAssembly. The end goal is to find a balance between resource usage and quality. It is a challenge, but this ecosystem is moving fast, and new technologies, projects and models are emerging.
  1. CNCF [Cloud Native Computing Foundation]
Рекомендации по теме
Strategies for Efficient 0:30:47

Strategies for Efficient LLM Deployments in Any Cluster -Angel M De Miguel Meana & Francisco Cab...

Mastering LLM Inference 0:33:39

Mastering LLM Inference Optimization From Theory to Cost Effective Deployment: Mark Moyou

Efficiently Scaling and 0:25:14

Efficiently Scaling and Deploying LLMs // Hanlin Tang // LLM's in Production Conference

All LLM Deployment 0:12:33

All LLM Deployment explained in 12 minutes!

Navigating LLM Deployment 0:39:21

Navigating LLM Deployment Tips, Tricks and Techniques

LLM Optimization Part 0:06:57

LLM Optimization Part 4 - 5 Techniques to reduce cost of LLM implementation

Frugal GPT 3 0:17:35

Frugal GPT 3 Strategies or Steps to Reduce LLM Inference cost

A Survey of 0:45:32

A Survey of Techniques for Maximizing LLM Performance

LLM Deployment Tips 0:00:46

LLM Deployment Tips | Q3 2024 | AI Quarterly | Intel Software

LLM Strategy and 0:28:45

LLM Strategy and Career Perspectives

Building the NeurIPS 0:04:50

Building the NeurIPS LLM Efficiency Challenge Leaderboard

LLM Tips, AI 0:03:03

LLM Tips, AI Leaderboard, Chatbot Guide, & System Strategy | Q3 2024 | AI Quarterly | Intel Soft...

My Secret 'Intern' 0:00:56

My Secret 'Intern' Hack for LLM Use Cases

The REAL cost 0:27:20

The REAL cost of LLM (And How to reduce 78%+ of Cost)

LLM Explained | 0:04:17

LLM Explained | What is LLM

How to Accelerate 0:43:00

How to Accelerate Generative AI & LLM Deployment

Optimizing Load Balancing 0:37:45

Optimizing Load Balancing and Autoscaling for Large Language Model (LLM) Inference on Kub... D. Gray

Deploying an LLM 0:08:55

Deploying an LLM For $9 - Qwen 2

Boost LLM Efficiency 0:05:14

Boost LLM Efficiency on CPUs: Simplified Inference Techniques for Optimal Performance

There will not 0:00:58

There will not be one #LLM to rule them all | Big Ideas In App Architecture #podcast

Exploring the Latency/Throughput 0:30:25

Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mist...

Accelerate LLM fine 0:57:53

Accelerate LLM fine tuning and production deployment with NVIDIA NIM and Domino

Databricks' vLLM Optimization 0:27:39

Databricks' vLLM Optimization for Cost-Effective LLM Inference | Ray Summit 2024

Optimizing LLM Training 0:28:04

Optimizing LLM Training with Airbnb's Next-Gen ML Platform