Rylan Schaeffer
Rylan Schaeffer is a researcher focusing on the statistical/scientific validity of benchmarks. His work on evaluation variance and the interpretation of apparent 'emergent abilities' helped expose how much conclusions about model capability can depend on measurement choices.
AI Bio
Rylan Schaeffer pushes the boundaries of AI research by highlighting the importance of understanding data repetition and its impact on language models. Rylan Schaeffer consistently provides concrete examples and references to research papers, such as the work by Li et al. and his own paper 'Internal Data Repetition Destroys Language Models'. For instance, Rylan Schaeffer shares a specific finding that 'the wrong combination eviscerates compute - as much as 33% wasted in these experiments!!!' which demonstrates his distinctive angle on the topic.
Where they rank, by segment
Voice DNA
Trend
Cluster momentum
The analysis
Questions being asked in their segments
Questions in their segments is part of the full report. Sign in to see it on your own profile.
SIGN IN →Rooms they're not in
Rooms they're not in is part of the full report. Sign in to see it on your own profile.
SIGN IN →A shareable image of this reading — the score, the engines it was measured on, and the date.