@nopatience haha ok and that's from Claude Sonnet which is (... checking marketing materials...) "an exceptional model when it comes to logic-based tasks and reasoning" 😆

Maybe what happens here is simply that the word "footloose" is uncommon in the dataset they fed into it, the dataset has no examples of sentences referring to the number of o's in that word, so it will spit out something for similar words?

Seems like it does not do "reasoning" at all, it's just another bullshit generator.