Uploading Models – DeployModel

This page will go through all the steps necessary for deploying any of your model into a production ready chargeable API. You can also find this fully working template and many more on Github here (opens in a new tab).

Prerequisites

Ensure Docker is installed and running on your system. If you haven't installed Docker yet, you can download it from the official Docker website (opens in a new tab).
Install our python package

pip install deploymodel

Prepare Your Version

Let us say that you have an AI model (in PyTorch, TensorFlow or any framework you like) in a file called model.py. For the sake of our example, we will use a model that classifies between cars and cats, according to a noise input. Here is the code:

model.py

from difflib import SequenceMatcher
 
 
class CarCatModel:
    def __init__(self) -> None:
        self._cat_template_noise = "meow"
        self._car_template_noise = "vroom"
 
    def forward(self, noise: str):
        car_similarity = SequenceMatcher(None, noise, self._car_template_noise).ratio()
        cat_similarity = SequenceMatcher(None, noise, self._cat_template_noise).ratio()
        if car_similarity > cat_similarity:
            return {"label": "car", "confidence": car_similarity}
        else:
            return {"label": "cat", "confidence": cat_similarity}
 
    def __call__(self, *args, **kwargs):
        return self.forward(*args, **kwargs)

Create and configure a file called deploy.py. In this example, we will place it in the same directory as our model.py file. This defines thin wrappers around your inputs, outputs and models. These wrappers allow for documenting your APIs. An example is provided below, feel free to adapt it according to your needs.

deploy.py

from deploymodel.io import Field, ModelInput, ModelOutput
from deploymodel import register
 
from model import CarCatModel
 
 
class CarCatInput(ModelInput):
    noise: str = Field(
        ...,
        title="Noise",
        description="Noise to be classified as either car or cat",
    )
 
 
class CarCatOutput(ModelOutput):
    label: str = Field(
        ...,
        description="The label output by the model (can be either Car or Cat).",
    )
    confidence: float = Field(
        ...,
        description="The confidence of the model in its prediction, between 0 and 1 for each item in the batch.",
    )
 
 
def init() -> CarCatModel:
    # This function is called once when the model is loaded
    # When using PyTorch/TensorFlow, you would load the model weights here
    return CarCatModel()
 
 
def handler(model: CarCatModel, input: CarCatInput) -> CarCatOutput:
    output = model(input.noise)
    return CarCatOutput(label=output["label"], confidence=output["confidence"])
    
 
if __name__ == "__main__":
    register({"handler": handler, "init": init})

💡

The deploy.py file is crucial as it defines how your model interprets input and generates output, which is essential for integrating your model with DeployModel.

Take your time to be exhaustive in describing each parameters.

We recommend that you commit deploy.py to your version control, this will save a lot of time for you to deploy future versions.

Build Your Image

We'll add 2 more files in the same directory as model.py: a requirements.txt and a Dockerfile, with the following content

requirements.txt

deploymodel

Dockerfile

# Only Debian/Ubuntu base images are currently supported
FROM python:3.10

WORKDIR /app/
COPY ./requirements.txt .
RUN pip install -r requirements.txt

# Setting the PYTHONPATH is mandatory
ENV PYTHONPATH=/app
COPY . .
WORKDIR /app
ENTRYPOINT ["python", "deploy.py"]

💡

Make sure that your base image is a Debian/Ubuntu image.

💡

Make sure that the environment variable PYTHONPATH is properly set (see here)

Your directory should now look like so:

project/
│
├── model.py           
├── deploy.py          
├── requirements.txt   
└── Dockerfile

Remember that you can find a more advanced example at here (opens in a new tab).

Make sure that your image builds and run properly by running the following from the directory where the Dockerfile is in:

docker build -t test .
docker run --rm test

If everything goes, well, you should see the following output:

Init /app/deploy.py:init Handler /app/deploy.py:handler registered successfully.

Upload Your Image

Head to deploymodel.com/model (opens in a new tab), create your model, and add a version to it. Copy the snippet provided, for instance:

Once this is done, run the command.

deploymodel push <token> -i <Dockerfile>

Replace <token> and <Dockerfile> by the appropriate values (token should be filled, and Dockerfile is the location of the file we created above). This could take a couple of minutes. Once this is done, the status should change to UPLOAD STARTED then to ACTIVE. You are now ready to deploy your model!

Deploy Your Model

Click on Deploy, specify the instance details and your model will be deployed.

What is DeployModel?Monetization