Your Own Llama 2 API on AWS SageMaker in 10 min! Complete AWS, Lambda, API Gateway Tutorial

Llama 2 from Meta has been out for a few weeks now and it’s a compelling alternative to ChatGPT for anyone looking to develop custom applications for their business or startup due to its commercial nature, large context window and open-source model.

I’ve created a short tutorial showing product developers how they can leverage AWS SageMaker, AWS Lambda, API Gateway, to deploy and create their own LLama 2 API for use on their own projects.

LLAMA 2 PLAYGROUND

CODE SNIPPETS

AI AGENTS COURSE SIGNUP

MUSIC
Chad Crouch Algorithms

[mamunrashid]

Thanks, Rob. That very helpful tutorial.
For some reason, I have been receiving "Hello from Lambda!"

[supermanish]

easily explained man thanks, my specific requirement is that i want to train the model periodicly with my own data how can i achieve this?? can u create a video around this query, thank you

[wasgeht]

Thanks! One question, if I use the jump start models from aws for example llama 2 and start with the fine tuning so then it's my own fine tuned model? and I can use it for my own use cases ?

[theAIjurist]

thank you very much for the tutorial, this was really helpful

[datadockter]

This is the best one so far. But can you please make one where how I can run these models on my windows.

[fadichehimi]

thanks a lot for this helpful demo!! appreciate all your efforts and notes!

[azcro-dev]

excellent! exactly what i was looking for. thxs

[yashinshekh]

this is amazing. thank you. exactly what i looking for

[SolidBuildersInc]

Thank you for your presentation. I clicked the Subscribe button, although I didn't delve into the video content. During your talk, I recall you mentioning the open-source LLM and discussing AWS pricing. This led me to prioritize a cost-effective solution that allows for scalability. Have you considered running an ollama model locally and setting up a tunnel with a port endpoint for a public URL? I appreciate any feedback you can provide." 😊

[cowabungatv]

great tutorial, keep up the good work !

[LuizAntonioJunior]

Thanks for this tutorial. One important thing we would like to know is: How much is it montlhy on AWS? Its kinda difficult to understand their price model.

[barasoft]

Thanks for this tutorial. Can we do all these using Amazon SageMaker Studio Lab? Can we find a guide for that somewhere? Thanks 🙂

[phishic]

thank you Rob for the tutorial, I really appreciate how you put it, so much clear and precise.
I am a student and need to deploy LLM in AWS for the Uni project, I want to try doing from my own AWS account with free trier and trying to understand how much would that cost if I only need to deploy it and check with a few inputs (if I am talking about Llama-2-7b-chat) and don't want to end up with 400USD bills. do you think just the deployment process would cost much?

[brittanymorris]

I'm still learning -- but why would i use sagemaker over bedrock? when do i pick one over the other? specifically for deploying LLMs?

[bqgezck]

Hey, thank you for the video. However LLama2 is not available in my SageMaker, so I guess your video is obsolete

[hrvojek]

I am new to AWS and Llamas, and this provided a great first insight. One question though - once we create a Domain, does it mean we will get charged immediately? I do not remember setting up an account and providing the credit card info, so I am a bit confused on how this is actually operating. Can you point me into some documentation which explains this? Thank you!

[sasukeuchiha-ckhy]

Can you train the AI using your own dataset, and can you integrate it into an existing website?

[nicolassuarez]

Oustanding! But what's the point of spending lots of money with SageMaker inestead of directly using Groq's api with LLama? Thanks!

[lauvindra]

Hi Great video btw. I would like to ask if it's possible to make concurrent request using Sagemaker. Will it drastically increase the cost or the costing is based on the hours of usage?

[Xplay]

rob i have a doubt why to use this where we can use models - ondemand without pricing for instance as base models we pay for only tokens