Inference Costs: Are We Nearing an Inflection Point?

With model sizes plateauing but deployment exploding, inference cost is starting to dictate architecture. We keep talking about quantization and batching tricks, but hardware efficiency seems stuck. Energy costs and datacentre density are also forcing real decisions on where to run large models. Are we hitting a wall where algorithmic improvements no longer offset infra overhead, or is there a shift coming in how we think about evaluation and dedicated silicon? Curious what others are seeing in production pipelines, and whether anyone has had to turn down workloads purely because of the price per token.