Secret Dates in System Prompts Undermine Language Model Evaluation
New research reveals that embedding hidden date information in system prompts can significantly skew language model evaluation results, as models may use temporal cues to infer which benchmarks they should perform well on. This is a subtle but serious evaluation integrity issue — if models are sensitive to date signals, benchmark scores may reflect overfitting to evaluation periods rather than genuine capability. For developers and researchers running their own evals, this is a practical warning to audit system prompts for any contextual signals that could leak information about the evaluation context. The finding also has implications for how public leaderboards should be interpreted, particularly for models with known training cutoffs close to benchmark publication dates. Reproducible, signal-clean evaluation design is now a more pressing concern than many teams currently treat it.
Read original source ↗Part of the 2026-10-10 briefing→