We run a multi-tenant inference service and autoscaling keeps biting us. Scale-up takes four to seven minutes to pull the image and warm
Cold starts are wrecking our inference SLOs
Replies
No comments yet — be the first to share your thoughts.
Log in
No account yet? Sign up