Good evening r/artificialIntelligence , With a friend of mine, I co-run a fairly small-potatoes astronomy channel on YT (we’re both PhD astrophysicists), and we’re experimenting with podcasting, general astronomy education, and a less hype / more science rigour format. A viewer of our channel (@Astraveo) sent us an AI-generated astrophysics manuscript hoping for it to be of high enough quality and interest to be submitted to a leading astrophysics journal. So, instead of breaking it down as a research paper submitted by a citizen scientist, we tried to peer review it as we would if we were officially refereeing the paper from a career astrophysicist. Here are our criteria: – – Does the Introduction describe the historical foundation of the field, and describe its advancement ? – What is the central science claim, is it original, and what would falsify it ? – Are the scientific methods and process described at a high-enough level for the experiment to be reproducible ? – Do the citations and references actually support the scientific statements being made ? – Is there a clear separation between results, interpretation and speculation ? What surprised us most, wasn’t that the LLM was able to write something coherent (of course they can now), but the ease with which such a research paper can “feel” rigourous while being hard to audit unless an expert really digs deeper and checks each claim in the argument chain. So, my discussion question here: If you were the editor or reviewer of a similar journal research paper, what should be the minimum standard for AI-assisted manuscripts ? Some discussion ideas: – should a disclosure of AI-assistance be mandatory ? – should there be automated checks for citation integrity ? – Are LLMs just another research tool that should be exempt (or more ‘relaxed’) from rules of scientific reproducibility ? I’d be happy to share what our referee checklist looks like, and summarize the paper’s failure points – without doxxing or dunking on our viewer’s submission. (if anyone wants the video link, I can drop it in the comments). An advanced draft of this post was run through M365 Copilot (version #:2.20260319.58.0) to improve its readability. submitted by /u/Doppler_kid
Originally posted by u/Doppler_kid on r/ArtificialInteligence