Smart RAG: Domain-Specific Fine-Tuning for End-to-End Retrieval

Have a question for a speaker? Drop them here:

Speakers:
Dr. Greg Loughnane, Founder & CEO, AI Makerspace

Chris Alexiuk, CTO, AI Makerspace

Join our community to start building, shipping, and sharing with us today!

Apply to our upcoming LLM Ops: LLMs in production course on Maven today!

How'd we do? Share your feedback and suggestions for future events.

[pshreyaan]

This is super interesting, I hope the channel blows up and we get to see more similar content. 👍

[chrisogonas]

Thanks Team, for that educative session. 👍

[micbab-vgmu]

Thank you the topic of RAG is very interesting.

[li-pingho]

this video saved my life amazing work!

[taylorfans]

You guys are the RAG masters ! Thank you for the informative videos.

[pavellegkodymov]

Thanks for the video, guys, most useful were definitely notebooks with comments of Chris. Thumb up, subscribed. But appreciate also information given by Greg, just seemed a bit high level or not explained deep enough/explained too complex. Probably better would be then to explain less but in more details, with examples. The aim I assume is not watch and think "or that guy knows a lot, although I don't understand anything". But actually to learn something. Straight and maybe not "nice" feedback, but hope it will help. You guys are doing great job sharing insides and helping others. I'm far behind with that so far))

Haven't reached so far a point where I need to fine tune. Working now mostly on retrieval step and different strategies like pre-filtering of text (key word search) before retrieval from vector store. But definitely fine tuning of embedding model might be one of solutions for me. As for now I'm struggling to "emphasize", give higher priority to some domain relevant words automatically inside the question over other regular non relevant ones like "in", "please" etc. To get chunks that are more relevant for answer within K chunks selected and increase chance to provide more relevant context to LLM. So thanks for hints and well done!

[alchemication]

Super interesting content. Thanks for posting!

I would say the piece, which I often miss is an actual example of using this thing (is it 1 model, 2 models, do we still use vector db?)
And also some discussion about the practical side of things. What if my data changes?
Nevertheless awesome work, cheers guys!

[sivi]

Awesome video again! These videos are blowing up!
My company has 200K+ pdfs ranging from 100 pages to 10000+ pages. Will this framework work for such enormous data? Wondering how long it will take to create the synthetic triples for millions of PDFs' chunks that would get created from 200K+ pdfs. Would love to hear your thoughts!!!

[ashritkulkarni]

Thanks Team, this a great video.
How do we now make queries from the dalm after training and finetuning?

[arkabagchi]

Great video on E2E RAG pipelines and where /when to fine tune (embedding model, the LLM itself, or retrieval models). I was wondering if you had a source or links to relevant literature that specifically talks about this E2E evaluation framework (Arxiv papers or something similar)?

Thanks a ton and keep up the work in this retrieval pipeline space. Knowledge augmented language models are going to be amazing.

[aswathmg]

Thank you for that nice video guys. You mentioned that there are open source models for generating synthetic data. Can you please suggest any?

[zd]

Quick question, if I have a query rewriting model that takes a user's input query into a simplified or a set of simplified sub queries to be sent to the retriever, I'm thinking can I leverage this framework to train the query rewriter, retriever, as well as the generator (LLM) end-to-end?

[nenjaplays]

Thank a lot Team! Question: After finetuning, how does one save the fine-tuned model to disk?

[eagle]

Dear can you share your keynote? The resolution is up to only 720.

[saka]

Great tutorial - thank you! However, a lot of useful information is posted in the chat window and it got lost when the tutorial ends. I don't know if there is a practical solution to this, unfortunately.

[mosca]

Can I use a custom model as a generator?

[peregudovoleg]

Hey guys.
At ~14:30 you talk about genrating answers with question-context pairs, but generate_answer func never calls the context, just the question. Wouldn't it be better to give llm question AND context? Since the question, I would assume, might sound like: "What Arthur decided to do at the end?" And without the context, llm might give suboptimal answer.
I have also not found a link to colab NB, did you give any?
Thanks for the great talk!

[nasiksami]

What open source platform/library can I use to create synthetic data, instead of using openIAI?

[nelohenriq]

Can you explain how to achieve the same without OpenAI?
Thanks in advance

[robertcringely]

Stop reading from a script, Greg, and ditch the goofy hat. It's distracting.