Ollama can run LLMs in parallel!

preview_player
Показать описание
In this video, we're going to learn how to run LLMs in parallel on our local machine using Ollama version 0.1.33.

#ollama #llms #llama3 #phi3

Рекомендации по теме
Комментарии
Автор

Thanks for the code and the information

arskas
Автор

Fantastic video and thanks for sharing!
A few notes for I already have ollama installed and I have to turn it off before running the commands.
1. Turn off the ollama on ubuntu or mac ` systemctl stop ollama.service` before running `OLLAMA_NUM_PARALLEL=4 OLLAMA_MAX_LOADED_MODELS=4 ollama serve`
2. Line 12 in your code will not work on some streamlit version as the API key is ` api_key="ignore-me"` but should be `api_key="ollama"` by default

mcast
Автор

Can you explain how to use this parallel capability with the ollama python library.

ajmalm
Автор

Can you show the Parallelism of Ollama through LangChain too? Thank you

arskas
Автор

How does this work with the docker version of Ollama? Can someone please help me here?

modx
Автор

I agree, would be great if you can provide the code for doing this in python 😊

tlubben
Автор

What IDE are you using please let me know.

karthikb.s.k.
Автор

Your all videos are great. Can you prepare few videos on LLM OS where we can use Ollama?

anilrajshinde