1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import requests
import base64
# Use this function to convert an image file from the filesystem to base64
def image_file_to_base64(image_path):
with open(image_path, 'rb') as f:
image_data = f.read()
return base64.b64encode(image_data).decode('utf-8')
# Use this function to fetch an image from a URL and convert it to base64
def image_url_to_base64(image_url):
response = requests.get(image_url)
image_data = response.content
return base64.b64encode(image_data).decode('utf-8')
api_key = "YOUR_API_KEY"
url = "https://api.segmind.com/v1/sdxl-controlnet"
# Request payload
data = {
"image": image_url_to_base64("https://segmind-sd-models.s3.amazonaws.com/display_images/sdxl-controlnet-input.jpeg"), # Or use image_file_to_base64("IMAGE_PATH")
"prompt": "American woman, smiling, waving, eyeglasses, casual clothing, sitting at desk, laptop, home office, indoor, daylight, modern decor, house plants, bookshelf, natural lighting",
"negative_prompt": "low quality, ugly, painting, extra fingers, deformed",
"samples": 1,
"scheduler": "UniPC",
"cn_model": "sdxl_canny",
"num_inference_steps": 30,
"guidance_scale": 6,
"seed": 760941192,
"controlnet_scale": 0.9,
"base64": False
}
headers = {'x-api-key': api_key}
response = requests.post(url, json=data, headers=headers)
print(response.content) # The response is the generated imageAttributes
Input Image
Prompt to render
Prompts to exclude, eg. 'bad anatomy, bad hands, missing fingers'
Number of samples to generate.
min : 1,
max : 4
Type of scheduler.
Allowed values:
Type of Controlnet
Allowed values:
Number of denoising steps.
min : 10,
max : 100
Scale for classifier-free guidance
min : 1,
max : 25
Seed for image generation.
min : -1,
max : 999999999999999
Controlnet Strength
min : 0,
max : 1
Base64 encoding of the output image.
To keep track of your credit usage, you can inspect the response headers of each API call. The x-remaining-credits property will indicate the number of remaining credits in your account. Ensure you monitor this value to avoid any disruptions in your API usage.
SDXL ControlNet
SDXL ControlNet empowers users with unprecedented control over text-to-image generation in SDXL models. By incorporating conditioning inputs, users can achieve more refined and nuanced results, tailored to their specific creative vision. It achieves this by introducing the concept of conditioning inputs, which provide additional information to guide the image generation process. This allows for precise control over various aspects of the final image. A key feature of SDXL ControlNet is its ability to leverage control images as conditioning inputs. These control images can include:
-
Depth maps: Encode spatial arrangements and distances within an image.
-
Canny edge maps: Capture outlines and edges, ideal for tasks like comic book art and architectural visualization.
-
Human pose keypoints (from OpenPose): Enable control over character pose, gestures, and dynamics in animations and character design.
-
Free-form sketches and scribbles: Allow for rapid concept iteration and exploration in tasks like concept art and children's book illustrations.
-
Edge Extraction : Allows for extracting edges from images to create a sketch-like representation.
Specific Applications of ControlNet
-
Canny ControlNet: Ideal for generating comic book art with bold outlines and ink-like strokes, or highlighting building structures and edges in architectural visualizations.
-
Depth ControlNet: Well-suited for populating virtual reality environments with realistic textures or showcasing products with accurate depth cues in product design.
-
Openpose ControlNet: Particularly useful for animating characters with precise poses in character animation or creating virtual fashion models for showcasing clothing in fashion design.
-
Scribble ControlNet: Enables rapid concept iteration through sketches and scribbles, ideal for tasks like concept art and children's book illustrations.
-
SoftEdge ControlNet: It’s useful for tasks like converting photos into sketch-like versions or enhancing existing images with soft edges.
Popular Models
