ML-at-Scale '23 - LLM Batch Inference with Determined

preview_player
Показать описание
Speaker: Corey Staten
In this talk we used Determined's Core API and Hugging Face Transformers to build and optimize batch inference workflows. We also discussed some advanced parallelization techniques, and showed how to achieve them using Determined's DeepSpeed integration. Warning: This session is code-heavy!
  1. Determined AI
Рекомендации по теме
ML-at-Scale '23 - 0:52:04

ML-at-Scale '23 - LLM Batch Inference with Determined

ML-at-Scale '23 Day 3:38:16

ML-at-Scale '23 Day 1

Enabling Cost-Efficient LLM 0:30:28

Enabling Cost-Efficient LLM Serving with Ray Serve

Faster and Cheaper 0:28:04

Faster and Cheaper Offline Batch Inference with Ray

Accelerate Big Model 0:01:08

Accelerate Big Model Inference: How Does it Work?

Bench AI, Inc. 0:00:58

Bench AI, Inc. YCombinator S23 Video

SBTB23:Oleg Avdeëv& Riley 0:32:18

SBTB23:Oleg Avdeëv& Riley Hun, Lesson learned from orchestrating large-scale GenAI, ML & dat...

Lakehouse to LLM: 0:25:55

Lakehouse to LLM: Solve Business Tasks with LLMs Reliably and at Scale

How to eat 0:00:16

How to eat Roti #SSB #SSB Preparation #Defence #Army #Best Defence Academy #OLQ

Epoch, Batch, Batch 0:03:29

Epoch, Batch, Batch Size, & Iterations

Integrating Generative AI 0:14:19

Integrating Generative AI Models with Amazon Bedrock

Deploy Gen AI 0:57:53

Deploy Gen AI at Scale with Snowflake

True ML Talks 0:34:02

True ML Talks #23 |Discover the MLOps and LLMs space at GitLab

This ML Project 0:09:56

This ML Project Gives You an Unfair Advantage

A friendly introduction 0:24:19

A friendly introduction to distributed training (ML Tech Talks)

How to Build 0:35:45

How to Build an LLM from Scratch | An Overview

Build enterprise-grade Q&A 1:01:26

Build enterprise-grade Q&A at scale with Open LLMs on AWS

State-of-the-Art Retrieval Augmented 0:33:04

State-of-the-Art Retrieval Augmented Generation at Scale in Spark NLP

The Future Of 0:23:42

The Future Of LLM Training Is Federated

Joscha Bach - 1:52:46

Joscha Bach - Why Your Thoughts Aren't Yours.

The Future of 0:00:56

The Future of Scalability in Machine Learning: Predictions and Challenges

[SDC23] AI & 0:24:50

[SDC23] AI & MLOps infrastructure for enterprise-grade LLMs

Deploying Many Models 0:25:42

Deploying Many Models Efficiently with Ray Serve

End To End 0:40:25

End To End Machine Learning Project Implementation Using AWS Sagemaker