vLLM Office Hours - Deep Dive into Mistral on vLLM - October 17, 2024

preview_player
Показать описание

During our special topic deep dives, we were joined by Mistral AI’s research engineer, Patrick von Platen, who shared insights into Mistral’s architecture choices and how to efficiently deploy Mistral's models on vLLM.

During the Q&A, we tackled audience questions on topics such as architecture redesign strategies, rotary position embeddings, vLLM support for ARM architecture, OpenAI Whisper, Seq2Seq support in v0.6.3, and more.

  1. Neural Magic
  2. vllm
  3. mistral
  4. deploy mistral's models on vllm
Рекомендации по теме
vLLM Office Hours 0:49:38

vLLM Office Hours - Deep Dive into Mistral on vLLM - October 17, 2024

vLLM Office Hours 0:56:09

vLLM Office Hours - FP8 Quantization Deep Dive - July 9, 2024

vLLM Office Hours 0:48:13

vLLM Office Hours - vLLM on AMD GPUs and Google TPUs - August 21, 2024

vLLM Office Hours 0:59:55

vLLM Office Hours - SOTA Tool-Calling Implementation in vLLM - November 7, 2024

vLLM Office Hours 0:52:35

vLLM Office Hours - Advanced Techniques for Maximizing vLLM Performance - September 19, 2024

Accelerating LLM Inference 0:35:53

Accelerating LLM Inference with vLLM

Fast LLM Serving 0:32:07

Fast LLM Serving with vLLM and PagedAttention

vLLM Office Hours 0:50:38

vLLM Office Hours - Model Quantization for Efficient vLLM Inference - July 25, 2024

vLLM: Easy, Fast, 0:23:33

vLLM: Easy, Fast, and Cheap LLM Serving for Everyone - Woosuk Kwon & Xiaoxuan Liu, UC Berkeley

vLLM Office Hours 0:53:19

vLLM Office Hours - June 20, 2024

Boost Your AI 0:10:54

Boost Your AI Predictions: Maximize Speed with vLLM Library for Large Language Model Inference

vLLM and PagedAttention 0:05:50

vLLM and PagedAttention is the best for fast Large Language Models (LLMs) inferencey | Lets see WHY

CUDA Mode Keynote 0:20:46

CUDA Mode Keynote | Lily Liu | vLLM

E07 | Fast 0:55:36

E07 | Fast LLM Serving with vLLM and PagedAttention

Llama 3.2 Deep 0:12:30

Llama 3.2 Deep Dive - Tiny LM & NEW VLM Unleashed By Meta

But what is 0:11:13

But what is DeepSpeed ? DeepSpeed vs VLLM

Exploring the Latency/Throughput 0:30:25

Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mist...

Deploy LLMs More 0:33:21

Deploy LLMs More Efficiently with vLLM and Neural Magic

Video Comprehension using 0:00:28

Video Comprehension using GenAI | Qwen 2 VL 2B #llm #imageprocessing #imagerecognition #vlm #qwen

Unlocking LLM Efficiency: 0:03:08

Unlocking LLM Efficiency: PagedAttention & vLLM Revolutionize Memory Management

vLLM on Kubernetes 0:27:31

vLLM on Kubernetes in Production

All You Need 0:10:30

All You Need To Know About Running LLMs Locally

What is Retrieval-Augmented 0:06:36

What is Retrieval-Augmented Generation (RAG)?

Accelerate Big Model 0:01:08

Accelerate Big Model Inference: How Does it Work?

Комментарии
Автор

Hi, great video, I would like to know if it is possible to use e5-mistral-7b-instruct in VLLM for embedding and completions with only one instance of VLLM?

micuentadecasa