Background Replace

Background Replace (BG replace) is SDXL inpainting when paired with both ControlNet and IP Adapter conditioning capable of generating photo-realistic images. This model offers more flexibility by allowing the use of an image prompt along with a text prompt to guide the image generation process. This means you can use an image prompt as a reference and a text prompt to generate desired backgrounds which look very close to real life.

This model is a improvement over the ControlNet Inpainting model.

Key Components of Background Replace

ControlNet helps in preserving the composition of the subject in the image by clearly separating it from the background. This is crucial when you want to maintain the structure of the subject while changing the surrounding area.

IP Adapter allows to use an existing image as a reference, and a text prompt to specify the desired background. For example, if you have a picture of a perfume bottle and you want to change the background to a forest with the help of a reference image (or IP image) of a forest, you could use a text prompt like “Perfume bottle placed on top of a rock, in a jungle”. The IP Adapter then guides the inpainting process to replace the background of the image in a way that matches the reference image. So, in our example, the output image will have the perfume bottle on a rock in a forest setting.

How to use Background Replace

1. Input image: Input an image of an object or subject, preferably with a white background.

2. Reference Image: For the reference image, input any image you want as the new background for the object/subject in the input image.

3. Advanced Settings: Start with the default settings (recommended) and iteratively tweak them based on your preference for the generated outputs. Check the recommended settings to have better control over the image outputs.

Recommended Settings

In addition to parameters like Guidance scale, scheduler, and Strength, we recommend keeping a close eye on these two parameters, as they greatly influence the final image output.

CN Weight: This is ControlNet Weight. This setting determines the degree of influence the ControlNet conditioning (Canny Edge Detection) has on the original image. A higher weight means a greater influence. It is always recommended to keep the CN weight high to preserve the composition of the object/subject in the input image.

IP Adapter Weight: This is IP Image Weight. This setting determines the degree of influence the Image Prompt (reference image) has on the original image. A higher weight means a greater influence and vice versa. The recommended value is 0.5, as it is neither too high nor too low. A high value can be counterintuitive, as colors from the image prompt will start creeping into the subject/object in the input image.

Background Replace in Pixelflow Workflows

Background replace is used in many background generation and replacement workflows in Pixelflow. It is ideal for product photography use cases in the e-commerce industry. Some of the workflows that utilize BG replace include:

1. Furniture background Generator: Generates new backgrounds (indoor and outdoor) settings for a furniture piece.

2. Shoe background Generator: Generates new outdoor backgrounds for running shoes.

3. Jewelry Background Generator: Generates new backgrounds for jewelry.