Ollama - Libraries, Vision and Updates

🕵️ Interested in building LLM Agents? Fill out the form below

👨‍💻Github:

⏱️Time Stamps:
00:00 Intro
00:11 Ollama
00:31 Python and JavaScript Intro
00:56 Vision Models Intro
01:22 OpenAI Compatibility Intro
02:12 Demo
05:30 Using the Python Library
07:31 Vision Models
10:08 OpenAI Compatibility

[RaspberryPi-gsts]

i love this series of introduction ollama!!! a lot!!!

[mukkeshmckenzie]

If they had an option to load multiple models at the same time (if there's enough ram/vram), it would be cool. The current workaround is to dockerize an ollama instance and run multiple of them on the same gpu.

[lucioussmoothy]

Thanks for pulling this together. Really like the /show /save capabilities. Suggests new ways of creating and updating model files.

[motbus]

Wow. I am impressed to find one useful AI related channel.
I mean you show things running with your code, you state real problems you find and you discuss your own results.
Please continue with that 🙏 and thank you very much!

[acekorneya]

would be awesome for some tutorial videos on how you make those automated tools would be awesome to know how to do things like that

[Leonid.Shamis]

Thank you for another very informative video. It would indeed be cool to hear more about using Ollama and local LLMs with AutoGen and for a fully local RAG system.

[changchoi]

Wow so cool how local LLMs are progressing! So many ideas - can't handle hahah

[dllsmartphone]

ollama is the nest bro. i use it witz web ui its amazing

[thehve]

Greater canter through the recent updates, have to say I am a fan of ollama and have switched to using it almost exclusively in projects now. Not least as it's easier for others in my team to pick up. Really short learning curve to get up and running with local LLMs.

[stephenthumb]

Essentially this is based on llama.cpp embedded in Go but stranglely cannot handle concurrency. Love ollama and use it a lot but to run it in a production setting you have to basically spin multiple ollama server each of which can take a queue. In other words a load balancer setup with niginx or something.

[attilavass]

Pls. create a video about hosting an LLM server with Ollama on Google Colab (free T4) available via API. That might be a cost efficient way of hosting "local" models.

[supernewuser]

I just noticed some of these updates yesterday and it let me simplify some bits of my stack and remove litellm. It's actually kind of scary how quickly all of this stuff is becoming commodity parts.

[kenchang]

I just saw on Matt Williams channel that Ollama now run on Windows natively. Just thought I'd mention it to you.

[Zale]

Great video! Can you please cover stanfordnlp's dspy, amazing library!

[aiexplainai]

great video as always! Would you consider to cover lepton ai? looks like a great way to host llm on local machine

[equious]

I serve a model with ollama and I hooked it up to a discord bot :D

[ShikharDadhich]

Ollama is Awesome however there are some minor issues with oLlama:
1. Single threaded, so can not run on server serving single url to team. It’s big issue, I don’t want everyone in my team install ollama in their machine.
2. With Stream response its not easy to create client app as the response is not same is OpenAI
3. CORS issue, so need a wrapping around the APIs, which means you need to install ollama and install api wrapper on every machine

[squiddymute]

can you actually stop ollama (linux) somehow ? or it runs forever and ever on the background ?

[IronMechanic]

Does ollama can working without internet connection when i'm using a local llm ?

[matikaevur]

heh, run
ollama run llama-pro:text "what are you"
.. about 10 times and confirm that i'm not going crazy, it's the model . . .
that thing is outputting it's fine-tuning data verbatim .. AFAIK