Forget Photoshop - How To Transform Images With Text Prompts using InstructPix2Pix Model in NMKD GUI

Playlist of Stable Diffusion Tutorials, #Automatic1111 and Google Colab Guides, DreamBooth, Textual Inversion / Embedding, #LoRA, AI Upscaling, Pix2Pix, Img2Img:

I am also showing how to install and use NMKD Stable Diffusion open source GUI.

Easiest Way to Install & Run Stable Diffusion Web UI on PC by Using Open Source Automatic Installer

How to Use SD 2.1 & Custom Models on Google Colab for Training with Dreambooth & Image Generation

Stable Diffusion - Dreambooth - txt2img - img2img - Embedding - Hypernetwork - AI Image Upscale

0:00 Introduction to new AI model InstructPix2Pix Learning to Follow Image Editing Instructions
0:44 What is InstructPix2Pix
1:31 Examples of InstructPix2Pix from official developers
2:47 Introduction to the NMKD Stable Diffusion GUI
3:38 How to download NMKD Stable Diffusion GUI
4:42 How to setup and install NMKD Stable Diffusion GUI
6:56 How to run NMKD Stable Diffusion GUI
7:09 How to generate text to image AI images by using NMKD Stable Diffusion GUI
7:26 How to open settings of NMKD Stable Diffusion GUI
7:48 How to use InstructPix2Pix in NMKD Stable Diffusion GUI
8:01 How to select which GPU or use CPU in NMKD Stable Diffusion GUI
8:22 How to load base image and instruct Pix2Pix to transform images
9:07 How to display both original image and transformed image in NMKD Stable Diffusion GUI
9:46 What are Prompt and image guidance and CFG scale of InstructPix2Pix image transformation
10:23 More detailed explanation of Image CFG weight and Text CFG weight
12:33 How to open logs / debug CMD window in NMKD Stable Diffusion GUI
14:01 Visual effects of different prompt and image guidance CFG scale
15:09 How to try multiple different prompts by using prompt queue feature of NMKD Stable Diffusion GUI
16:14 Different prompt examples of InstructPix2Pix
17:28 How to upscale transformed images into bigger resolution by using AI upscale algorithms

Abstract
We propose a method for editing images from human instructions: given an input image and a written instruction that tells the model what to do, our model follows these instructions to edit the image. To obtain training data for this problem, we combine the knowledge of two large pretrained models---a language model (GPT-3) and a text-to-image model (Stable Diffusion)---to generate a large dataset of image editing examples.

InstructPix2Pix: Learning to Follow Image Editing Instructions
If you're not getting what you want, there may be a few reasons:

Is the image not changing enough? Your Image CFG weight may be too high. This value dictates how similar the output should be to the input. It's possible your edit requires larger changes from the original image, and your Image CFG weight isn't allowing that. Alternatively, your Text CFG weight may be too low. This value dictates how much to listen to the text instruction. The default Image CFG of 1.5 and Text CFG of 7.5 are a good starting point, but aren't necessarily optimal for each edit. Try:
Decreasing the Image CFG weight, or
Increasing the Text CFG weight, or
Conversely, is the image changing too much, such that the details in the original image aren't preserved? Try:
Increasing the Image CFG weight, or
Decreasing the Text CFG weight
Try generating results with different random seeds by setting "Randomize Seed" and running generation multiple times. You can also try setting "Randomize CFG" to sample new Text CFG and Image CFG values each time.
Rephrasing the instruction sometimes improves results (e.g., "turn him into a dog" vs. "make him a dog" vs. "as a dog").
Increasing the number of steps sometimes improves results.
Do faces look weird?
The Stable Diffusion autoencoder has a hard time with faces that are small in the image.
Try: Cropping the image so the face takes up a larger portion of the frame.

[SECourses]

Please join discord, mention me and ask me any questions. Thank you for like, subscribe, share and Patreon support. I am open to private consulting with Patreon subscription.
NMKD is best GUI / UI option for AMD and Intel GPU users.

[FernandoMtnezFoto]

This is amazing! Extraordinary tool! Thank you for the video!

[우아한잉아]

"You are running this program from a path that contains special characters(신인웅)
Please move it to a path without special characters and try again" I tried to run according to the video, but the following window appears. Is there a solution?

[ganeshr]

How do I use image2image inpaint in Stable diffusion to swap face from my trained model using another civicai chekpt?

[spinninglink]

is there an updated version of how to install this now?

[magedmohammed]

Which is the better option for upscalling the method you used or gigapixel Ai?

[arisleonel]

I have a mac. Where can I got to pay someone to do this for me? What terms do I search for in upwork or the like?

[monicahernandez]

[01-30-2023 11:08:09] ModuleNotFoundError: No module named 'diffusers.onnx_utils'

im trying to convert the model that comes in the original download version of NMKD sd-v-1-5-fp16.ckpt, i didnt add any new model. Thanks again for your help!

[ZeroCool]

I tried it, but it doesn't work good on images that aren't 512 res. right?

[WeisenbergTKMrWhite]

Its not possible to run it with 6GB VRAM right? (i have gtx 1060 6gb gddr5)
Can you provide a tutorial on how to run it in colab through automatic1111 or gradio?

[faroazandeen]

How can you install it in the Colab Version?