I keep coming back to the argument that "emergence" in LLMs is largely an artifact of discontinuous metrics like exact-match accuracy, and that switching to token-level or continuous measures makes the curves smooth. It is a clean claim, but I rarely see it stress-tested outside a handful of tasks. Has anyone tried replicating this on newer model families or on reasoning benchmarks where the jump looks genuinely sharp? I am also curious whether the same critique applies to in-context learning curves. If the effect is mostly metric choice, that changes how we should report scaling results, and I would like to see more negative or mixed replications.
Are emergent abilities just a metric artifact?
Replies
No comments yet — be the first to share your thoughts.