The tested LLMs fared much worse, though, when the Apple researchers modified the GSM-Symbolic benchmark by adding “seemingly relevant but ultimately inconsequential statements” to the questions

Good thing they’re being trained on random posts and comments on the internet, which are known for being succinct and accurate.