You could also use your own frameworks to deploy your models. You could follow the instructions:
- Dockerize your model server.
- Upload your Docker image to a public Docker registry (e.g. Docker Hub).
- Create the deployment with the custom image, choose
Otherframework, and set the
portin the advanced features.
You could enable autoscaling for your deployment. The autoscaling feature will automatically scale up and down the number of replicas based on the request load. You could set the minimum and maximum number of replicas.