AI

  • Hugging Face Papers — “Schrödinger’s Code Repository: Have LLMs Learned SWE-bench or Memorized It?” introduces SchrodingerRepo, which transforms test repositories at evaluation time (problem-statement rewording, namespace remapping, file-layout reordering, functionality-preserving code rewrites) and finds these transformations cut SWE-bench Verified Pass@1 by 6.0-14.4 points, indicating coding agents lean heavily on memorized repository-surface cues rather than genuine understanding.