The EASIEST Local LLM App (OpenAI-compatible API)!!!

Meet Jan - Bringing AI to your Desktop Element hero heading
Open-source ChatGPT alternative that runs
100% offline on your computer.

🔗 Links 🔗

❤️ If you want to support the channel ❤️
Support here:

🧭 Follow me on 🧭

[alx]

These guys (or the guy) who are doing this are just professionals - with their approach to plan all the work ahead. This product is doomed to be great. Love them

[shawnvines]

Excellent video. I've been using LM Studio for awhile but I'm going to try this now. Thank you very much for sharing!

[tvwithtiffani]

This video made me think of something ....these apps always tout openai compatible api server as a benefit. But prompts are married to their models. What i mean is that going from a system that relied on a more capable model (openai) to a system that is relying on a local lm, prompts outside of very basic prompts will need to be rewritten to accommodate for the local models short comings. So in hindsight idk how much a openai compatible api server is an actual benefit.

[PallaviChauhan]

that's great, I can finally use models other than chatgpt 4 or 3.5 for the very first time!

[davidw]

Just have installed it on the weekend :) love it, agree it's underrated! And great video!

[thelateknights]

Really interesting video! Jan looks great, but I'm unsure why this is in the Local RAG Tutorial playlist? Don't you say specifically that they do NOT have RAG capability?

[abdelkaioumbouaicha]

📝 Summary of Key Points:

📌 Jan is a tool for running local large language models (LLMs) on your computer. It offers ease of use, extensions, reusability, and openness.
🧐 Jan is easily downloadable and installable on various platforms. It recommends models based on your machine's specifications and RAM capacity.
🚀 Jan is praised for its openness, with a transparent product roadmap and open-source code for privacy and security.
🌐 Jan offers an API server compatible with OpenAI chat completions, allowing users to build applications or chatbots easily.
💬 The conversation history in Jan can be used for fine-tuning LLMs or generating data to improve models.
📣 Jan is recommended as a user-friendly, open-source tool for running local LLMs.

💡 Additional Insights and Observations:

💬 "Jan can be easily downloaded and installed on Linux, Windows, and Mac."
🌐 The company behind Jan provides a transparent product roadmap on their website.
📊 No specific data or statistics were mentioned in the video.
🌐 The speaker mentions exploring other options like GP4All with RAG.
🌐 The speaker expresses curiosity about Jan's future sustainability and revenue model.

📣 Concluding Remarks:

Jan is a user-friendly tool for running local large language models (LLMs) on your computer. It offers ease of use, extensions, reusability, and openness. With its transparent product roadmap and open-source code, Jan provides a reliable and secure option for running LLMs locally. The ability to fine-tune models using conversation history and the compatibility with OpenAI chat completions make Jan a versatile tool. However, the speaker raises questions about Jan's future sustainability and revenue model. Overall, Jan is recommended as a valuable tool for running local LLMs.
Generated using TalkBud

[TomM-po]

Another tool in the toolbox along with Text Generation Web UI and LM Studio. Add to that a couple of agentic frameworks a dozen leading LLMs, and I'm overwhelmed 😄

Can you do a video discussing which combinations work best together?

[Ari_Alur]

Thanks for showing this! Very helpfull!

[__________________________]

I will try Jan soon because it sounds interesting open-source, free and api server

[lhxperimental]

Just found out that it is local UI app. Not a web based app that you can run on a server and access the UI from a browser. That sucks because I like to run things on a beefy headless desktop/room heater with RTX4090 while I sit comfortably in another room.

[alexxx]

Hopefully a better alternative to LM Studio

[malleswararaomaguluri]

Can you suggest me which embedding and llm model is best for rag using gpt4all. Thanks. I am windows user. That's why i tried with gpt4all but not ollama

[sekolahstata]

for version to install in Cpanel, are there capable to install in cpanel ?

[tarunkoundinya]

Have you came across any frameworks that have mlx integration? because we have apple silicons but we are not really using it to our fullest potential… if not, we have to create one.

[stsifui]

i am a medical professional actually intrested in fine tuning mixtral 8x7b with the data i generated in chat gpt through custom instruction to fine tune the open source model any guidence is appreciated

[metanulski]

I think a big topic is "can the model speak my language". There are no Videos about this topic.

[nhtna]

when u say local, with 1.xb parameters, what would be the size that you need on your local laptop, along with the memory/gpu etc?

[HyperUpscale]

Amazing video with a lot of details, comparison and demonstration!

[ajaykolekar]

Why no support for AMD GPU's? So much beautiful UI, just lack of this....