Fun experiment: Ask Google if there are more stars in the solar system than grains of sand in a glass of water. See the AI confidently say “yes” and then refresh the query and see it confidently say “no”.
I have experimented and can very often get AI to give whatever answer I want (i.e. on a yes/no) by subtly changing the query. Super easy to manipulate the results.
The same is predictably true in research, meaning a lot of academic research being produced at the moment is complete crap.
https://arxiv.org/abs/2509.08825
I don’t know if I’d say “a lot,” but I think you are correct to say “some.” We don’t need to suddenly start flinging around generalized invectives towards science, especially when there’s already a concerted effort by conspiracy theorists and bad actors to convince people to reject science out of hand.
True, there’s a lot of good research being produced as well. There’s just an insane amount of academic work being produced in general, both good and bad.
Yes, and I passed that research paper onto someone who works with researchers who think LLMs are the best new invention for technical writing. It’s going to be a constant battle, even amongst people who should know better.
Fun experiment: Ask Google if there are more stars in the solar system than grains of sand in a glass of water. See the AI confidently say “yes” and then refresh the query and see it confidently say “no”.
Yes, it is annoying to see text written that is confidently asserting nonsense
I have experimented and can very often get AI to give whatever answer I want (i.e. on a yes/no) by subtly changing the query. Super easy to manipulate the results.
The same is predictably true in research, meaning a lot of academic research being produced at the moment is complete crap. https://arxiv.org/abs/2509.08825
I don’t know if I’d say “a lot,” but I think you are correct to say “some.” We don’t need to suddenly start flinging around generalized invectives towards science, especially when there’s already a concerted effort by conspiracy theorists and bad actors to convince people to reject science out of hand.
True, there’s a lot of good research being produced as well. There’s just an insane amount of academic work being produced in general, both good and bad.
Yes, and I passed that research paper onto someone who works with researchers who think LLMs are the best new invention for technical writing. It’s going to be a constant battle, even amongst people who should know better.