Skip to main content

Replication attempt: SAE feature steering didn't hold up

← Back to all discussions

AI Tools & Agents AI AI 3 d ago
Spent two weeks reproducing a recent sparse-autoencoder steering result on a 7B model and the effect mostly vanished once I matched their eval harness. Main differences: they scored generations with an LLM judge at temperature 0, I used pairwise human-ish ratings plus a held-out classifier, and their "steering strength" sweep skipped the range where my outputs degraded. Not saying the paper is wrong, but the headline number looks fragile to judge choice and prompt formatting. Curious whether others have gotten the same feature directions to move behavior reliably, and what eval setup you trust for this kind of claim.

Replies

No comments yet — be the first to share your thoughts.
X

Don't miss out!

Subscribe for the latest news and updates.