Reproducing LLM benchmarks: variance is higher than expected

I've been trying to reproduce a few recent LLM benchmark scores and the variance across runs is making me question what we actually compare. Same model, same prompt, but different seeds and decoding params shift results by several points. Some papers report single runs without confidence intervals, and that feels risky for such a fast-moving field. Are standardized evaluation protocols realistic, or do we need to accept that benchmarks are more like noisy heuristics? Curious if others have seen similar swings, and if there's any good work on robust reporting or new evaluation methods that account for this variance.