All LLM Deployment explained in 12 minutes!

preview_player
Показать описание

This video shows how to build your private LLM API Server for Pennies. The cheapest perhaps way to deploy your LLM and serve it as an API using Salad.

In this concise yet comprehensive guide, we dive into the process of building your own private Large Language Model (LLM) API server on a budget. Discover the most cost-effective strategies to deploy your LLM and serve it as an API using Salad. Perfect for developers, hobbyists, and tech enthusiasts looking to leverage AI without breaking the bank!

🔍 What You'll Learn:

Step-by-step process to set up your private LLM API Server
How to deploy your LLM cost-effectively
Tips and tricks to optimize performance and reduce costs
Whether you're a beginner or an experienced developer, this video will provide you with the tools and knowledge to efficiently deploy your own LLM. Don't miss out on these insights – watch now and take your tech projects to the next level!
Рекомендации по теме
Комментарии
Автор

Nice. What happens with data though. Do they do data retention/logging like openai does? They log all data going into the model and all data coming out of the model

jeffsteyn
Автор

I use Modal for temporary resource timesharing instead of always-on deployment.

jsalsman
Автор

Can you make a video on how to fine tune a model to be used with ollama please 🙏

jayalmeida
Автор

0.32 per hour for what? Time of returning data? or time running on server? That was not really clear. Openai's api is cheaper I think

ArtisanTony
Автор

Very cool
What I would like to do is AI API that can use my local git repo and something like StarCoder or similar, integrated with VS Code or similar dev editor

isrbillmeyer
Автор

Can we ssh connect and train models or it is only for deployment?

satpalsinghrathore
Автор

Hi brother, is this deployment able to serve parallel users at a time, as I've seen some docs saying ollama deployment can complete only 1 request at a time and puts other queries in a queue.
Thanks

akhilreddygogula
Автор

I was interested in providing them some iron until I saw the BT firewall advice. fun thing anyway

twobob
Автор

Small aside...on the joke..Tunnel vision is another name for peripheral vision loss, and with Musk's boring company (Not sure if Zuckerberg has any vision issues, it clarifies :-) the joke is quite good!

pavguy
Автор

it is asking 50$ even before
starting .

sidiocity
Автор

Its not user friendly as Runpod is ... i hope they will improve fast

valm