Exploring the fastest open source LLM for inferencing and serving | VLLM

Hello everyone welcome to our latest video. Today we will explore the open source inferencing and serving LLM, VLLM. Our founder Vishnu Subramanian will take you through on what is VLLM and also how to use them.

Connect with us on social media:

Connect with Vishnu:

[bernard]

This was a nicely paced and clear tutorial. Thank you. Liked and subscribed.

[Akshatgiri]

Super useful. Thanks for breaking it down.

[dineshgaddi]

Thank you for sharing this information.

[YajuvendraSinghRawat]

Its a wonderful videa, clearly and concisely explained.

[kaiwalya_patil]

An excellent one! Thank you so much for sharing.
Any idea about the possibility of fine tuning my own LLM(like Llama/Mistral), uploading back to HF and the put it into production using VLLM?

[fxhp]

hey i also have an AI channel, i tried mistrals model and it didnt finish its execution and looped over the input forever, i had slightly better luck with the instruct version. did you ever get mistral to work?

[alecd]

Thanks for your video. It is interesting.
I am new to LLM and one question to ask.
When you run JarvisLabs in your demo, does it mean you are running a server running locally to provide API endpoint?
Please advise