K8s for LLM inference: overkill or essential?

We're running a small GPU fleet for internal LLM tooling, and the team is split on Kubernetes. Some want it for autoscaling and multi-tenant isolation, others argue it's too much operational overhead for a few models. Add in the complexity of GPU scheduling, cold starts, and the cost of node pools that aren't fully utilized. Curious how others handle inference workloads in production — are you fully on K8s, or using managed serverless GPU options? Also, what's your take on investing time in custom autoscalers versus relying on the cloud provider's tooling?