About AI for math, here is a nice plot twist from the paper “Early science acceleration experiments with GPT-5”, https://arxiv.org/abs/2511.16072
“We were very excited to have an exact solution to a question which we had been curious about for years, especially because one direction of the proof was due to an AI. However, we soon learned that the same tight bound (as well as a different proof of the matching upper bound) had appeared on Arxiv nearly 3 years previously in the short paper [Alo24]. In fact, GPT-5’s proof of the lower bound was exactly the same as Alon’s! Thus, it appears that GPT-5 reproduced Alon’s proof and passed it along to us without realizing its source. As it turns out, a fresh attempt at querying GPT-5 Pro was able to recover this source, as depicted in Figure II.4.
Our experience illustrates a pitfall in using AI: although GPT-5 possesses enormous internal knowledge and the capability to locate even more using the internet, it may not always report the original information sources accurately.”
(Section II.3.1)
I wish to be kind to these authors, and I won't name them, but I can't imagine they hadn't heard of that issue.
> "GPT-5 reproduced Alon’s proof and passed it along to us without realizing its source"
This kind of highlights the core functionality of such systems: it's a plagiarism machine. It's truly "useful" if your goal is to throw away authorship information and present something as new, produced by you or your "intelligent" system. The obvious main usecase is fraud.
For anyone interested in honesty and facts, much better to use a search engine that could point to the source.
