I have been trying to reproduce a few of the "model organism" misalignment training runs that get cited constantly. Two of three gave clearly different behavior at the same seed and hyperparameters, and the emergent misalignment effect mostly vanished when I changed data ordering in the SFT mix. Curious whether others have hit this. If these artifacts are this brittle, how much weight should safety evals place on them? Would appreciate pointers to systematic replication efforts, or negative results people were reluctant to publish.
Do misalignment model organisms actually replicate?
Replies
No comments yet — be the first to share your thoughts.