How to Precisely Modify Local Areas in Stable Diffusion

Although SD generates images quickly and with decent quality, sometimes the AI can be difficult to control. For instance, when generating multiple images at once, there’s always a part that doesn’t meet your expectations! At this point, we need to utilize the local re-painting feature in SD! So how can we control the details in the drawings using SD? Let’s look at these two methods.

Image-to-Image Mode

Local Repainting

Suppose you used SD to generate an architectural rendering, and you feel that the front lawn is a bit empty and want to add something. Actually, you can directly modify it locally using SD.

We send the image to the image-to-image mode. If you want to modify the image directly, just open the image-to-image mode and drop the image in.

Brush Settings for Masking in the Area You Want to Modify

Enter the keywords you want to generate. For example, we want to generate some outdoor chairs, etc.

Just click to add!

Or you can add some flowers and plants as well.

However, this method has a problem; the mask needs to be applied manually. If you want to modify a specific material of the building, you need a very precise mask, and this method may not work well. You will need another tool!

Inpaint Anything

One-Click Color Channel Generation

Precise Control

First, we need to install this plugin. We can search for ‘inpaint anything’ in the extensions and install it with one click (it won’t show here because I already installed it).

Alternatively, you can enter this URL to install it.

https://github.com/geekyutao/Inpaint-Anything

After installing the plugin, restart the web UI, open the inpaint anything window, select a model, and click download!

Then you can import the previously generated rendering into the plugin.

Click to run segment anything, and you can generate the channel map with one click!

Use the brush to click on the outer wall of the building. Different colors represent different corresponding materials, and once selected, it gets cut out, then click to create a mask.

If the selection is not complete after creating the mask, you can click to expand the mask selection, and the area will increase.

If the system recognition is incomplete, you can also click ‘Trim Mask by Sketch’ and then click ‘Add Mask by Sketch’ to supplement it.

Afterwards, select ‘Only Mask’, click to get the mask, and the mask will be created! We have extracted the outer wall of the building.

Then click to send to image-to-image re-painting, and we return to the image-to-image mode.

Then click ‘Send to Image-to-Image Repainting’, and we return to the image-to-image mode. At this point, you can also choose to modify the large model and load lora, or load controlnet for control, input suitable keywords, and the other steps are just like the regular SD image generation mode.

For example, we want to change the outer wall to a glass curtain wall. One click can generate multiple options for you to choose from.

Of course, if you want better results, you still need to choose the appropriate large model and lora. This requires everyone to try more!

So, with this plugin, SD is very controllable. Now you can’t say AI is dumb! It’s incredibly intelligent!

If you haven’t installed the SD package yet, we can provide it below.

It’s time for our benefits! This time we offer the following benefits!

-SD Chinese Prompt Plugin-

Click the public account card

Send the code directly

To obtain

(No tricks, no need to forward)