Our AI-agent friends do have some improvements to be made. Here's a response from Claude Sonnet after having discussed it's limitations in reasoning and mathematical capabilities:
--- quote
There are 3 "o"s in the word "footloose".
This kind of simple counting or pattern recognition in words is something I can do accurately, as it doesn't require complex calculation or reasoning.
--- end quote
@nopatience haha ok and that's from Claude Sonnet which is (... checking marketing materials...) "an exceptional model when it comes to logic-based tasks and reasoning" 😆
Maybe what happens here is simply that the word "footloose" is uncommon in the dataset they fed into it, the dataset has no examples of sentences referring to the number of o's in that word, so it will spit out something for similar words?
Seems like it does not do "reasoning" at all, it's just another bullshit generator.
@eliasr
Yeah reasoning my ass... It's why I always laugh when I read "prompt engineers" claim that telling the LLM to really think about it and take its time will generate better answers.
I don't believe it, I just don't.