Try-On Diffusion

Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on

API

If you're looking for an API, you can choose from your desired programming language.

POST

import requests
import base64

# Use this function to convert an image file from the filesystem to base64
def image_file_to_base64(image_path):
    with open(image_path, 'rb') as f:
        image_data = f.read()
    return base64.b64encode(image_data).decode('utf-8')

# Use this function to fetch an image from a URL and convert it to base64
def image_url_to_base64(image_url):
    response = requests.get(image_url)
    image_data = response.content
    return base64.b64encode(image_data).decode('utf-8')

api_key = "YOUR_API_KEY"
url = "https://api.segmind.com/v1/try-on-diffusion"

# Request payload
data = {
  "model_image": image_url_to_base64("https://segmind-sd-models.s3.amazonaws.com/display_images/model.png"),  # Or use image_file_to_base64("IMAGE_PATH")
  "cloth_image": image_url_to_base64("https://segmind-sd-models.s3.amazonaws.com/display_images/cloth.jpg"),  # Or use image_file_to_base64("IMAGE_PATH")
  "category": "Upper body",
  "num_inference_steps": 35,
  "guidance_scale": 2,
  "seed": 12467,
  "base64": False
}

headers = {'x-api-key': api_key}

response = requests.post(url, json=data, headers=headers)
print(response.content)  # The response is the generated image

RESPONSE

image/jpeg

HTTP Response Codes

200 - OKImage Generated

401 - UnauthorizedUser authentication failed

404 - Not FoundThe requested URL does not exist

405 - Method Not AllowedThe requested HTTP method is not allowed

406 - Not AcceptableNot enough credits

500 - Server ErrorServer had some issue with processing

Attributes

model_imageimage *

Input Image.

cloth_imageimage *

Cloth Image

categoryenum:str *

What type of clothes

Allowed values:

num_inference_stepsint ( default: 25 ) Affects Pricing

Number of denoising steps.

min : 20,

max : 100

guidance_scalefloat ( default: 2 )

Scale for classifier-free guidance

min : 1,

max : 25

seedint ( default: -1 )

Seed for image generation.

min : -1,

max : 999999999999999

base64boolean ( default: 1 )

Base64 encoding of the output image.

To keep track of your credit usage, you can inspect the response headers of each API call. The x-remaining-credits property will indicate the number of remaining credits in your account. Ensure you monitor this value to avoid any disruptions in your API usage.

Try-on Diffusion

TryOnDiffusion, a virtual try-on AI technique can realistically superimpose clothing onto a person's image despite variations in body shape and pose. By warping the garment image and blending it seamlessly with the person's image. It is designed to address two challenges in virtual try-on. One is preserving the garment or clothing details and the other one is adapting to body pose and shape changes.

Key Components of Try-on Diffusion

TryOnDiffusion uses a system called Parallel-UNet. The parallel-Unet has two parts: Person-UNet and Garment-UNet.

Person-UNet analyzes a picture of a person with some added noise to capture detail.
Garment-UNet focuses on the image of the clothing that needs be to overlayed on the person.

To make the clothes fit perfectly, TryOnDiffusion uses a technique called "implicit warping." This lets the garment-UNet adjust the clothing's shape on the fly, while the person-UNet seamlessly blends it onto the person's image. This two-UNet teamwork with implicit warping is what allows TryOnDiffusion to create such realistic virtual try-on experiences.

How to use Try-on Diffusion

Input image: Provide an image of a person.
Cloth Image: Upload a clothing image, preferably with a white or transparent background. Ensure the image contains only the clothing piece.

a. Upper body: This involves overlaying clothing items such as t-shirts, tank tops, shirts, jackets, etc.,

b. Lower body: This involves overlaying lower body clothing items like pants, trousers, skirts, shorts, etc

c. Full Body: It involves overlaying a complete dress on a person's image.

Try-on Diffusion in Pixelflow Workflows

Try-on Diffusion is used in many virtual try-on workflows in Pixelflow. It is ideal for virtual try-on use cases in the e-commerce industry. Some of the workflows that utilize Try-On Diffusion include:

Full body virtual try-on: Superimposes any clothing (upper, lower and full body) onto a person’s image.

Other Popular Models

sdxl-img2img

SDXL Img2Img is used for text-guided image-to-image translation. This model uses the weights from Stable Diffusion to generate new images from an input image using StableDiffusionImg2ImgPipeline from diffusers