Ollama: The Easiest Way to RUN LLMs Locally

In this video, I will show you no-code method to run open source LLMs locally. In this easiest way, we will run Mistral-7B in Ollama and serve it via API.

LINKS:

Timestamps:
[00:00] Intro
[00:29] Ollama Setup
[02:22] Ollama - Options
[04:14] Ollama API

[Shaunmcdonogh-shaunsurfing]

This is fantastic. Congrats to the team who put this together

[whitneydesignlabs]

Very cool. So few videos talk about the API in Ollama. This is a very important function.

[NLPprompter]

mind blown look how simple that is! i love this channel.

[philipsnowden]

Great vid! Would love to see how to integrate an Ollama model, through their API, to Flowise.

[Shogun-C]

Are there plans to integrate LocalGPT with Ollama?

[positivevibe]

Another great video! Thanks for sharing! Will wait till they release the Windows version.

[adriantang]

Great sharing and another great video.

[TechnicalTerry]

I love the simplicity of this setup! Is there any way to import my own documents for it to reference?

[trobinsun]

Thank you very much. How to connect it to my own data only ?

[Bcs-Mohtisham]

Sir, can we run quantized version and make inference fast with any method like vllm?

[davidelks]

I was wondering if i could run mistral model on 16gb ram and 4gb vram is it possible?

[SonnySaint]

Hey bro amazing video .Can i integrate on Swift?

[nufh]

Hi, I'm just star to learn about the Machine Learning. What is the lowest spec that can I have to run this as a hobby for starter.

[MrMoonsilver]

Running on wsl 2 only my cpu was used. Is this expected on wsl or am I missing something? (Drivers etc are installed)

[MaharshiPandyaJanam]

What are the specs of your computer in terms of RAM?

[positivevibe]

Also, are these tools safe?
Like absolutely no data leak? What would you recommend using it while connected to wifi or not connected?

This is not only about this one, it is also about whisperer, whisper, text-gen Webui, and many many other offline AI tools!
I would highly appreciate your input/thoughts!
Thanks

[cudaking]

@engineerprompt tried to use with mistral 7b however i am not sure what quantize version since i tried ro compare with the question like the pond of lillies and door push and pull it failed. So can we download the veraion we like and link it or is there alternative way.

[MichealAngeloArts]

I wonder if we could run this under (WSL 2) in Windows since it supports Linux? and does it utilize the machine's GPU(s) if exists

[Nallu_Swami]

I dont get enough token generation speed, how to increase that ?

[picklenickil]

Correct me if im wrong but wsl2 is supported on win11 init?