MiniGPT-4 - Vicuna LLM with AI Vision - GPT-4 open source alternative 🔥

Required Links:
MiniGPT-4:

❤️ If you want to support the channel ❤️
Support here:

[pratiksitapara]

Hello Sir, thank you very much for such videos.
Could you please make a video on evaluating LLMs, no one is making videos on that. Can you explain lm-evaluation-harness repo

[texasmedialaw]

Regarding the installation instructions for this program, could you provide us with a video tutorial on how to install it locally because the demo is down?

[Infinifiction]

thanks, great breakdown of function and like you I don't believe it until I've tested it myself 😊 Look forward to more content.

[christopherchilton-smith]

Well, this is going to be interesting, I'm definitely gonna see if I can run this locally. I wonder if Vicuna is smart enough to use tools. Vicuna + plugins 😍

[profitbridge]

Hi, is it possible to run autonomous gpt without openai help?

[MaJetiGizzle]

I think if you lower the temperature you could reduce the number of visual hallucinations.

[Dex_M]

Hey guys i think this is a great idea that i came up with, it's basically a creative way of using the tools available to create something creative, this is my theory:

Oh this is so interestingly insane, this is actually a clever way of making something creative, but the problem is that it uses Vacunea which is trying to achieve 90% of Chatgpt so it's cloning what Chatgpt does and that is... it's not trained on human input, rather it's trained on it's own input for it's data base to generate results, i'm not well informed about how it works but that's how i vision it.

Now, if we have something like Open assistant(currently the last time i checked it, it's bad but it's still getting better), which is basically like Chatgpt but it's trained on human input for it's database + something like MiniOpen assistant that uses Open assistant Language model, and you input let's say a picture of something that's in your mind for example a guy falling from the sky and a dragon behind it and tell it to describe the image first then generate let's say a story a poem or something based on the image and so using this way you want to make your story into different instances of images; we should probably get something humanly creative instead of Minigpt-4 while using this method i described but it heavily relays on Open assistance. and you can always refresh the the result as many times as you want -- generate paragraphs on the image -- until you are satisfied.
Now if this all works then you can train the Minniopen assistant model on different story telling styles, ways, to get something that's told close to the way it's told for example in "Berserk" mange (rest in piece "Kentaro Miura"). Or if you are good at writing you can only use this method to get inspiration and adjust or something else anyway my point is this could be huge.

The other down side is that it relays on images, so you need to generate a rough image in your mind, the easiest way is on stable diffusion, and with how much stable diffusion evolved now it's possible to generate what you picture in your mind roughly;
So then, can't you generate a bulk of images, the images don't have to be perfect they only need to resemble what you imagine in your mind, take different instances of the quick animation that you created and work on it slowly. I feel like everything that i've said could be theoretically automated so you go from :


This following method is only for generating main events


(1) create a robotic story using chatgpt ----> (2) split the story into small paragraphs and into sentences. ----> (3) give each sentence to Chatgpt and make rules that it should remove words such as " and, or, the" and add commas between each words and to also include words such as masterpiece beautiful...ect (my point is, try to make those sentences from the generated robotic story usable in stable diffusion) (4) give every sentence to Stable diffusion and it will generate an image based on the sentence (you can also add some general negative prompts to your webUI) (5) generate 1 image or 3 images (6) give the image to MiniOpen assistant to generate a story based on the image, you set it to describe the image first then to make a story (7) Give the end result to Chatgpt-4 to rate it, you can make as many images as you want for each sentence of the robotic first story and give it to minigpt to make as many stories for one image for Chatgpt4 to rate (8) take the only good rated story for each sentence and put them together in a text notepad. (9) give everything to Chatgpt-4 and tell it to connect everything using proper grammar and linking words that are appropriate, and you can tell it to rate the end result and see if there is anything that chatgpt-4 thinks is kinda lacking on needs some final touches.
so in order to create something creative in my opinion you need to use Chatpgt-4, Stable diffusion, a bitter and creative Minigpt; Chatgpt-4 again to assess and assemble the whole mess.



And for creating dialogue between characters, you want something like Chatgpt4-x-alpaca running on the oobabooga text-generation webui and using Customised characters, for better dialogues between characters.

[Aaron-itil]

Thanks, Vicuna is really impressive!!!

[dikshitsingh]

Bhaiya, could you make a video explaining how to train our model like that by own from scratch using cloud machine to training to deploying on hugging face

[swannschilling]

Does it run on the text generation webui? 😅

[GRANATE]

The license on miniGPT4 says:
This repository is under BSD 3-Clause License. Many codes are based on Lavis with BSD 3-Clause License here.

ChatGPT says I am allowed to use it in a commercial project. Is that true?

[shouldb.studying]

Did you see ilya's talk at the Stanford vc thing the other day? I love open source but I can see the benefit of the most competent models being closed just so you've got something to call upon to quash any misaligned openagi that comes along

[rccarsxdr]

I appreciate you sharing the new LLM tutorial. I'd like to ask if you could explain how Google Colab works as it's simple to test and comprehend.