TensorFlow Serving

+ Gradient

How to use Gradient and

TensorFlow Serving

together

Using TensorFlow Serving to deploy models on Gradient.

TensorFlow Serving is an open source model serving system, designed for production environments. TensorFlow Serving is closely associated with the TensorFlow project but can be used to deploy models developed in other frameworks.

Gradient includes a pre-built TensorFlow Serving container out of the box which is updated regularly. Alternatively, customers can use a customized version of TensorFlow Serving by using their own Docker image hosted on a public or private Docker registry.

Deploying models with TensorFlow Serving

When creating a Deployment, you can select the prebuilt image or bring your own custom image. These options are possible via the web UI, the CLI, or defined as a step within an automated pipeline.

Selecting the prebuilt TensorFlow serving image when creating a deployment

When using the CLI, the command would like something like this:

gradient deployments create \  

--name "my deployment" \
--deploymentType TFServing \
--imageUrl tensorflow/serving:latest-gpu \
--machineType P5000 \
--instanceCount 2
...

Learn more about the Gradient TensorFlow serving integration in the docs.