LangChain - Using Hugging Face Models locally (code walkthrough)

Load HuggingFace models locally so that you can use models you can’t use via the API endpoints. This video shows you how to use the end points, how to load the models locally (and access model that don’t work in the end points) and load the embedding models locally.

My Links:

Github:

[insightbuilder]

Keep up the great work. And thanks for curating the important HF models that we can use as alternate for paid LLMs. When learning new tech, using the free LLMs can provide the learner a lot of benefits.

[bandui]

Thank you! I am a newbie in this area and your vid´s are helping me a lot to get a better picture of the current landscape.

[prestigiouss]

Great tutorial. I need to train a model on some private company documents that aren't publicly released yet and this looks like it could be a big help to me. Subbed!!

[steevd]

Nice video. Im trying to connect an LLM and use Unity 3d as my interface for STT and TTS with 3d characters. I just found a tool that enables connex to a LLM on huggingface which is how I discovered that you need a paid endpoint with GPU support to even run most of them, I kinda wish I found this video when you posted it. Very useful info.

[luisxd]

Amazing video, everything was well explained, I needed it, thank you so much!

[Chris-senc]

Thanks for the video. Is there any way to get an example using the lang chain JavaScript library? I am new to this area, and I think many developers would have a node versus a python background

[tushaar]

Great video sam, i don't know how i missed this

[yves]

i am using huggingface model chavinlo/alpaca-native
however, when i use those embeddings with this model

pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
max_length=248,
temperature=0.4,
top_p=0.95,
repetition_penalty=1.2
)

local_llm =

my output is always only 1 word long. can anyone explain this?

[DanielWeikert]

I tried to store the YoutubeDownloader loads in FAISS using HuggingFace Embeddings but the LLM was not able to do the similarity search. Colab finally ran into timeout.
Can you share how to do this instead of using OpenAI? With OpenAI I had no issues but like to do it with HF Models instead e.g. Flan
br

[binitapriya]

Hi Sam, Is there any way to generate question answer from a given text in a .txt file and save those questions answers in another .txt file with the help of free huggingface model?

[sakshikumar]

saved me from hours of debugging and research! thanks a ton

[markomilenkovic]

If we cannot afford to get A100, what's the cheaper option you would recommend to run these? I understand the models differ in size also. Thanks Sam.

[jzam]

Thanks for the content!! Is there a way to run a HuggingfacePipeline loaded model using M1/M2 processors on Mac? How would one set that up?

[intelligenceservices]

is there a way to compile a huggingface repo to a single safetensors file? (compiled from a repo that has the separate directories: scheduler, text_encoder, text_encoder_2, tokenizer, etc...)

[azzeddine]

How can the ready-made projects on the platform be linked to Blogger blogs? I have long days searching to no avail

[brianrowe]

Stupid question, so I'll take a link to another video/docs/anything. Which Python version, cuda version, pytorch is the best to use for this work? I see many using python 3.9 or 3.10.6 specifically. The pytorch site recommends 3.6/3.7/3.8 on the install page. Then the cuda version 11.7 or 11.8 - it looks 11.8 is experimental? Then when I look at my nvcc output its says 11.5, but my nvidia-smi says cuda Version 12.0 .. head explodes... I'm on Ubuntu 22.04. I will google some more, but if someone know the ideal setup.. or at least the it works setup.. I appreciate it!!! Thank you

[luisxd]

I have a problem, when I use low_cpu_mem_usage or load_in_8_bit,
I get an error about I need to install xformers,
When I install xformers, I get an error I need to install accelerate,
When I install accelerate, I get an error I need to install bitsandbytes,
And so on: einops accelerate sentence_transformers bitsandbytes
But finally, I got an error *NameError: name 'init_empty_weights' is not defined*
I don't know how I can solve this error and why it happens, could you help me please?

[AdrienSales]

Excllent tutorial, ad so weel explained. Thanks a lot.

[atharvaparanjape]

How can I load the model for some time later, once I download it on the local drive

[anubhavsarkar]

Hello. Can you please make a video on how to use the SeamlessM4T HuggingFace model with langchain ? Particularly for text to text translation. I am trying to do some prompt engineering with the model using Langchain's LLMChain module. But it does not seem to work ...