Run a model from Python

Learn how to run a model on Replicate from within your Python code. It could be an app, a notebook, an evaluation script, or anywhere else you want to use machine learning.

🐍 Check out an interactive notebook version of this tutorial on Google Colab.

Install the Python library

We maintain an open-source Python client for the API. Install it with pip:

pip install replicate

Authenticate

Generate an API token at replicate.com/account/api-tokens, copy the token, then set it as an environment variable in your shell:

export REPLICATE_API_TOKEN=r8_....

Run a model

You can run any public model on Replicate from your Python code. Here's an example that runs stability-ai/sdxl to generate an image, then returns the URL of the image:

import replicate
 
output = replicate.run(
  "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
  input={"prompt": "an iguana on the beach, pointillism"}
)
print(output)
 
# ['https://replicate.delivery/pbxt/VJyWBjIYgqqCCBEhpkCqdevTgAJbl4fg62aO4o9A0x85CgNSA/out-0.png']

Using local files as inputs

Some models take files as inputs. You can use a local file on your machine as input, or you can provide an HTTPS URL to a file on the public internet.

Here's an example that uses a local file as input to the LLaVA vision model, which takes an image and a text prompt and input and responds with text:

import replicate
 
image = open("my_fridge.jpg", "rb")
output = replicate.run(
    "yorickvp/llava-13b:a0fdc44e4f2e1f20f2bb4e27846899953ac8e66c5886c5878fa1d6b73ce009e5",
    input={
        "image": image,
        "prompt": "Here's what's in my fridge. What can I make for dinner tonight?"
    }
)
print(output)
# You have a well-stocked refrigerator filled with various fruits, vegetables, and ...

Using URLs as inputs

URLs are more efficient if your file is already in the cloud somewhere, or it is a large file.

Here's an example that uses an HTTPS URL of an image on the internet as input to a model:

image = "https://example.com/my_fridge.jpg"
output = replicate.run(
    "yorickvp/llava-13b:a0fdc44e4f2e1f20f2bb4e27846899953ac8e66c5886c5878fa1d6b73ce009e5",
    input={
        "image": image,
        "prompt": "Here's what's in my fridge. What can I make for dinner tonight?"
    }
)
print(output)
# You have a well-stocked refrigerator filled with various fruits, vegetables, and ...

Handling output

Some models stream output as the model is running. They will return an iterator, and you can iterate over that output:

iterator = replicate.run(
  "mistralai/mixtral-8x7b-instruct-v0.1",
  input={"prompt": "Who was Dolly the sheep?"},
)
for text in iterator:
      print(text)
 
      🐑
      D
     olly
      the
      sheep
      was
      the
      first
      mamm
     al
      to
      be
      successfully
      cl
     oned
      from
      an
      adult
      cell
     ...

Next steps

Read the full Python client documentation on GitHub.