Author: Joy Deng
Ever since AI has been invented, it has been implemented through a multiarray of creative purposes, including art and literature. Scientific research is also inherently creative. Though AI can be an incredibly useful tool for discovery, there are dangers of treating it as a scientific partner.
When AI creates art, it doesn’t go through the specific steps of learning lighting, color theory, experimenting with brushstrokes, making failed art and iteratively improving the process– it just takes in a large model of art and tries (but often fails) to compose similar artwork. AI research is similar. If a researcher doesn’t go through the tedious process of doing preliminary literature research, grasping the concepts, experimenting with different methods, reexamining results, iteratively improving experimental design, and brainstorming applications of research, instead just jumping into the creation of the “perfect” experiment that will yield “ground-breaking results”– the results are also shallow, and perhaps even artificial.
AI was created by the scientific community, so it holds a different role in researcher’s hearts than other creative applications like literature and visual arts. It was created by computer scientists but is also currently replacing coder’s jobs.
There are usually four big stages in research: literature reviews (looking at the current state of research in), experimental design, conducting the experiment, data analysis and interpretation.
Literature review “AI as Synthesizer” tools are imaged to be able to read through articles and summarize the current state of the field with a focus point on what the researcher wants to study.
I somewhat disagree with this application. Though I do agree with AI being an explainer of the current field, I think that there is great value in reading journal articles and pulling key information from them by ourselves– there is a broader view and it actually compels our brains to understand and work with the information.
Messeri and Crockett published an article in Nature that discusses the topic of “Artificial intelligence and illusions of understanding in scientific research.” They classified the role of AI in research into four archetypes:
Study design “AI as Oracle” tools are imagined as being able to objectively evaluate, and summarize massive scientific literatures, helping researchers to formulate questions in their project’s design stage.
I also mostly disagree with this approach. Again, I think that the tedious brainstorming and reading-journal-article stage should be when inspiration comes bit by bit. AI can be used as a tool to show possibilities, but the researcher should still attempt to validate the ideas through broad literature search.
Data collection “AI as Surrogate” is hoped to allow scientists in generating accurate stand-in data points, including as a replacement for human study participants, when data is otherwise too difficult or expensive to obtain.
I strongly disagree! AI should never generate data points (in most natural science experimentation and engineering validation) because the whole point of experimentation is to see if your predictions (or hypothesis) line up with the actual data. If an AI modelpredicts the result, what is the point of conducting the experiment? If the researcher’s work involved bettering/implementing AI models, that would be a different discussion.
Data analysis “AI as Quant” tools seek to surpass the human intellect’s ability to analyze vast and complex datasets.
I agree with this one application of AI to scientific research. When there is mass data collection, analyzing the data can become extremely tedious. AI can be used to draw graphs but not used to draw full conclusions about the study.
Peer Review “AI as Arbiter” applications aim to objectively evaluate scientific studies for merit and replicability, thereby replacing humans in the peer-review process.
I most strongly disagree with this application of AI. AI can be used to categorize different articles and assign labels, but not to evaluate merit. AI is known to be racist, propagating dialect prejudice and race-based medicine, according to Nature. This might be because lots of AI models are fed publicly available data (many which can be racist), and AI is oftentimes trained to identify correlation, not causation. Furthermore, USC researchers find bias in up to 38.6% of ‘facts’ used by AI.
Messeri and Crockett warn against treating AI applications from these four archetypes as trusted partners, rather than simply tools, in the production of scientific knowledge. Doing so, they say, could make scientists susceptible to illusions of understanding, which can crimp their perspectives and convince them that they know more than they do.
Another big danger of AI in research occurs when AI is viewed as more reliable than researchers. Mike Cummings says that when that happens, it creates a monoculture of knowers, in which AI systems are treated as an authoritative, and objective knower in place of a diverse scientific community of scientists and engineers with varied backgrounds, training, and expertise. Science is not isolated from the rest of this messy, imperfect world.
Research and discovery is an inherently creative practice– exploring what we don’t understand, recognizing the issues in the world, synthesizing different information across fields. There is no inherently objective truth (as supported by the theory of relativity, haha), and completely entrusting our creative juices and groundbreaking discoveries to a learning model will not achieve it. Instead, we should see AI as what it is– a tool– and use it to support data analysis but not replace creativity.
Works Cited
Cummings, Mike. “Doing More, but Learning Less: The Risks of AI in Research.” YaleNews, 7 Mar. 2024, https://news.yale.edu/2024/03/07/doing-more-learning-less-risks-ai-research. Accessed 20 June 2025.
Gruet, Magali. “‘That’s Just Common Sense’. USC Researchers Find Bias in up to 38.6% of ‘Facts’ Used by AI.” USC Viterbi | School of Engineering, 26 May 2022, https://viterbischool.usc.edu/news/2022/05/thats-just-common-sense-usc-researchers-find-bias-in-up-to-38-6-of-facts-used-by-ai/. Accessed 20 June 2025.
Hofmann, Valentin, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. “AI Generates Covertly Racist Decisions about People Based on Their Dialect.” Nature 633 (2024): 1–8. https://doi.org/10.1038/s41586-024-07856-5. Accessed 20 June 2025.
Omiye, Jesutofunmi A., Jenna C. Lester, Simon Spichak, Veronica Rotemberg, and Roxana Daneshjou. “Large Language Models Propagate Race-Based Medicine.” Npj Digital Medicine 6 (2023): 1–4. https://doi.org/10.1038/s41746-023-00939-z. Accessed 20 June 2025.