Run any AI model remotely for free on google colab

Say goodbye to having not enough compute power to run your dream ai model! 🚀
We use Ngrok, to expose our ollama server from the colab server to the internet. Ngrok is a software which allows you to create secure tunnels and expose local ports to the internet.

🔧 What is Ollama?
Ollama is an open source tool that simplifies the installation process of Llama2 and other large langue models (LLMs). It provides an easy-to-use interface for installing llms on your system, without requiring you to manually configure
any dependencies or settings. With Ollama, you can quickly and easily get started with Llama2 in just a few minutes! 🚀

📔 Jupyter Notebook

📝 Installation Steps

In this video, I'll walk you through the step-by-step process of installing ollama on google colab with a python jupyter notebook. From downloading the tool to running your first remote AI, I'll cover it all! 💻

🎬 Watch and Learn

Whether you're a seasoned AI user or just getting started, this video is perfect for anyone looking to install and use LLMs remotely without any privacy concern.. So sit back, relax, and let me
guide you through the process of using google colab with Ollama! 😊

00:00 - 00:42 Introduction
00:43 - 03:21 google colab coding
03:22 - 04:14 start your jupyter notebook
04:15 - 06:34 link your local machine to the remote host
06:35 - 07:10 Outro


🔗

#googlecolab #ollama #ollamaai #privateAI #chatgpt #privategpt

[FilipeBento]

Great stuff!!

Ngrok is now asking for auth -- solved this by adding

await asyncio.gather(
run_process(['ngrok', 'config', 'add-authtoken', '<my token>'])
)

before:

await asyncio.gather(
run_process(['ollama', 'serve']),
run_process(['ngrok', 'http', '--log', 'stderr', '11434']),
)

[SethuIyer]

Thank you so much. I was killing my Intel mac with the LLM questions xD. This gives a good rest for it.

[techwithmarco]

If you want to learn more about ollama.ai, head over to my initial video about it :)

[dmistclesgee]

great stuff bro, keep them coming, thanks again.

[mobilesales]

Tell me how can I add Tele-FLM-1T local llm model but directly install in Google colab and how host on server using Google colab and how can I put those address in any framework I mean how to configure it plz plz kindly tell me instructions plz I

[iamderrickfoo]

This is awesome stuff! Would like to know after this up can we connect this to Webui or Anythingllm?

[thoufeekbaber]

Thank you.

I could run this succesfully in the terminal, but how can access the model or the collab through jupyter notebook instance?

[renega]

Hi amazing stuff!
Is there a way to connect the ngrok to jupyter notebook? Thanks!

[jeffsanaraujo]

That’s a fantastic video! Do you know if Ollama has OpenAI API compliant endpoints? So we could use Google Colab as a “Backend-as-a-Service” for some time in our chatbots :) One way I saw people doing is to create a long audio (like 12 hours of audio), loading it in the Google colab, and giving it a play, it’s a silence audio. It seems to work to keep the session opened for more time.

[CharlesDubois-fp]

How can I make this work with the ollama library in a python script? This works well when typing the prompts directly in the terminal, but my script still seems to run on my local instance.

[QHawk]

*Great! Thanks, can you do it with kaggle?, and with a local notebook/VSC?* Any update to this ?

[ralfrath]

I have win 10. How can I start the model?

[barskaracadag]

Hi, I am jsut curious what is gonna happen once collab kicks us from using the gpu. Restart it all?

[Shivam-biuo]

how do i save the progress, because everytime i run it, it downloads the model all from the start?

[yanncotineau]

i got a 403 forbidden error, but replacing
run_process(['ngrok', 'http', '--log', 'stderr', '11434']) with
run_process(['ngrok', 'http', '--log', 'stderr', '11434', fixed it for me.

[pathsvivi]

Thanks for the video. One question though, how can I avoid downloading the language models every time I run Colab notebook? Can I save Ollama and its models in Google drive and retrieve them when running the notebook?

[aryanflory]

hey, how to the export step on windows?
I have the ollama installed

[jameschan]

Please help if I use windows PC desktop, how can I open terminals like MAC?

[mellio]

but can't run stable diffusion this way?

[bennguyen]

I imagine it's costly to run LLMs.. is there a limit on how much Google Colab will do for free?

I'm interested in creating a Python application that uses AI.. from what I've read, I could use ChatGPT4 Assistant API and I as the developer would incur the cost whenever the app is used.

Alternatively, I could host a model like Ollama, on my own computer or on the cloud (beam cloud/ Replicate/Streamlit/replit)?

As a 3rd option, could Google Colab work in my situation? Is OpenAI's Assistant API totally different from the API to programmatically interact with llama2, mistral, etc?