Motion Control SVD

Motion Control SVD is an innovative deep learning framework that breathes life into static images. By intelligently managing both camera and object motion, it empowers creators to achieve precise animation effects.

Playground API Pricing

API

If you're looking for an API, you can choose from your desired programming language.

POST

import requests
import base64

# Use this function to convert an image file from the filesystem to base64
def image_file_to_base64(image_path):
    with open(image_path, 'rb') as f:
        image_data = f.read()
    return base64.b64encode(image_data).decode('utf-8')

# Use this function to fetch an image from a URL and convert it to base64
def image_url_to_base64(image_url):
    response = requests.get(image_url)
    image_data = response.content
    return base64.b64encode(image_data).decode('utf-8')

api_key = "YOUR_API_KEY"
url = "https://api.segmind.com/v1/motionctrl-svd"

# Request payload
data = {
  "input_image": image_url_to_base64("https://segmind-sd-models.s3.amazonaws.com/display_images/motion-control-svd/motionctrl_ip.png"),  # Or use image_file_to_base64("IMAGE_PATH")
  "fps": 7,
  "motion": "CW",
  "motion_bucket_id": 127,
  "cond_aug": 0.02,
  "seed": 985445,
  "decoding_t": 1,
  "maintain_aspect_ratio": True,
  "motion_speed": 1,
  "base64": False
}

headers = {'x-api-key': api_key}

response = requests.post(url, json=data, headers=headers)
print(response.content)  # The response is the generated image

RESPONSE

image/jpeg

HTTP Response Codes

200 - OKImage Generated

401 - UnauthorizedUser authentication failed

404 - Not FoundThe requested URL does not exist

405 - Method Not AllowedThe requested HTTP method is not allowed

406 - Not AcceptableNot enough credits

500 - Server ErrorServer had some issue with processing

Attributes

input_imageimage *

Input image

fpsint *

Frames per second for the video

min : 5,

max : 30

motionenum:str *

Motion type

Allowed values:

motion_bucket_idint *

Motion bucket ID

min : 127,

max : 255

cond_augfloat *

Conditional augmentation

min : 0,

max : 1

seedint ( default: 985445 )

Seed for random number generation

decoding_tint ( default: 1 )

Decoding time

maintain_aspect_ratiobool ( default: true )

Maintain aspect ratio

motion_speedfloat ( default: 1 )

Speed of the motion

min : 0,

max : 10

base64boolean ( default: 1 )

Base64 encoding of the output image.

To keep track of your credit usage, you can inspect the response headers of each API call. The x-remaining-credits property will indicate the number of remaining credits in your account. Ensure you monitor this value to avoid any disruptions in your API usage.

Motion Control SVD

The Motion Control SVD is a sophisticated deep learning framework developed to generate motion-controlled animations from static images. By leveraging cutting-edge neural network techniques, this model transforms static imagery into dynamic motions, allowing precise control over various parameters to achieve the desired animation effects. Motion Control SVD effectively and independently manages both camera motion and object motion. Existing methods either focus on one type of motion or do not distinguish between the two, limiting their control and diversity. The models architecture and training strategy are designed to address the unique properties of camera and object motion.

How to Use Motion Control SVD?

Input Image: Upload the static image (PNG, JPG, or GIF) that you want to animate.
FPS: Set the frames per second (FPS) to determine the smoothness of the animation.
Motion: Select the type of motion (e.g., up, down, left, right, zoom in, zoom out etc).
Seed: Input a seed value for reproducibility or randomize the seed.
Generate: Click on the "Generate" button to create the animated image.

How to Fine-Tune Video Outputs with Motion Control SVD?

The model offers several parameters that can be fine-tuned to achieve specific animation effects:

Motion Bucket ID: This parameter helps control the motion style. Higher values typically result in more complex motion patterns.
Conditional Augmentation (Aug): Set this value to introduce variations based on conditional inputs. Higher values make the model more sensitive to input variations, producing diverse animations.
Decoding Time: This setting defines the time required for motion decoding. Longer decoding times result in smoother animations but require more computational resources.
Maintaining Aspect Ratio: Enable this option to preserve the original aspect ratio of the image, avoiding any distortion during animation.
Motion Speed: Adjust this value to control the speed of the motion. Higher values lead to faster animations, while lower values produce slower, more nuanced motion.

Use Cases

Entertainment and Media: Enhancing motion graphics, creating lively animations for characters, and generating dynamic visual effects for video content.
Advertising: Producing eye-catching animations for digital advertisements and promotional material.
Education: Developing illustrative animations for educational content, making learning interactive and engaging.
Social Media: Creating unique and engaging visual content to enhance social media presence and attract followers.

Other Popular Models

sdxl-img2img

SDXL Img2Img is used for text-guided image-to-image translation. This model uses the weights from Stable Diffusion to generate new images from an input image using StableDiffusionImg2ImgPipeline from diffusers

fooocus

Fooocus enables high-quality image generation effortlessly, combining the best of Stable Diffusion and Midjourney.

faceswap-v2

Take a picture/gif and replace the face in it with a face of your choice. You only need one image of the desired face. No dataset, no training

codeformer

CodeFormer is a robust face restoration algorithm for old photos or AI-generated faces.