The philosopher's stone and the chemist's catalyst: Thoughts on LLM usage for (applied math) research

As all contemporary think-pieces, this is about artificial intelligence; more specifically, the use of it in scientific research. I must first expose my general bias against it, which will be quite apparent in my writing. My bias, however, should not undermine the constructive use of my thoughts. On the other hand, there are plenty of opinions relying on dubious studies that likely cannot be replicated, e.g., that LLMs reduce ability to think critically, and that students may over-rely on them despite their benefits, etc. I must second say that I will use the terms LLM, AI, and models interchangeably; the nuances between these do not detract from my overall message.

All thoughts and opinions here are my own; they do not reflect my employer, collaborators, or my professional work.

At the same time, I see many researchers surrender to it, suggesting that there is a spectrum of use while staunchly asserting that no sane and accomplished academic is on the "zero" end of this scale. I also see more advanced forms of surrender in the form of over- or complete-reliance on Agents, with assertions that research will soon be controlled by the bots. I will avoid, for the most part, linking to social media posts that I personally find infuriating. These opinions, in my view, are patently false. I do not fully buy into the fact that it will absolutely destroy our future as academics---I cannot and I will not, though this is perhaps unrealistic idealism.

Nevertheless, I would like to make a tale of two discussions about the use of AI in research by using a formal mathematical thought process. In numerical analysis, we often wish to compare "the ideal result" with "the result we can realistically get" by splitting the argument, i.e., triangle inequality: I thus split the discussion into comparing "the ideal result" with "the best result we could possibly get", then compare this "best result" with "the realistic result." In the context of generative AI, then, I will call the former comparison the philosopher's stone (we wish to conjure into existence the exact answer to a scientific problem). The latter comparison I suggest as a simple chemist's catalyst. The catalyst, of course, does not perform alchemy, but can speed up reactions and may yet be perfected with further experimentation and research. The reason I suggest this breakdown is that it allows me separate the consequences of using AI into the philosophical/pedagogical versus practical/possibly transient.

The Chemist's Catalyst

I will start first with the catalytic argument. Any catalyst is clearly tailored to the application/reaction in question. In other words, your mileage may vary, and this advice may or may not hold for you. Our catalyst will, over time, be improved toward its best possible state. LLMs will gradually accumulate diminishing returns, though we have obviously seen enthusiasm for ongoing research into alternative methods for generative AI. As the returns diminish, then, we will be left with some AI that is nearly optimal given its constraints (of architecture, data, compute resources....), possibly making these arguments uninteresting or moot. I will give several borderline-coherent thoughts for this transient case, then.

First, for the love of God, please check your bibliography by hand. I don't care how good your LLM is, I don't care how "busy" you purport to be. You need to check it by hand. It improves your chances of getting accepted and it takes maybe thirty minutes to make sure everything is consistent. You might yet be able to tailor your references better, hemming them according to the message of the paper. On a converse note, when performing literature review, don't just ask a generic chatbot for advice on a topic. Consider using tools created specifically for this which reference verifiable papers; I have heard nice things about the Google scholar labs tool (though I am convinced that this is their cheapest LLM wrapped around a bag of decision trees). Other choices abound.

Second, check your "tedious plotting code" that you propose LLMs should write. I have seen several cases of people showing plots that poorly represent or completely falsify data. Needless to say, I do not think highly of these "researchers." See later on coding writ-large.

Third, everything I say in this essay is intended specifically for the act of research and writing. While I think there should be concerns, partly addressed later, about the larger usage of LLMs, I am reasonably convinced that they can make a very productive tool for one-off projects. For example, creating a nicer website, exploring different coding projects only tangentially related to research, etc. The important term here, though, is "one-off". If you will build on it substantially, or it will change substantially, in the course of your research, please refer to the remainder of these thoughts.

Fourth, you should read papers for yourself before delegating to AI (more on this later). It is natural, and perhaps productive, to desire a second opinion for a paper that you are reading. Unless AI substantially improves in the coming years, however, it will miss subtle points, oversimplify ideas, or misrepresent the findings. You should try to read the paper critically, take notes (I recommend Zotero reader, which has improved substantially in recent years), and understand the mechanisms/findings before delegating to AI similar responsibilities. In some sense, this is the only way you can get "independent" samples of the paper's messaging. If you only deign to read papers that AI thinks is "worth it," you will suffer from selection bias and be exposed to several dangers: the sycophantic nature of LLMs, hidden prompt injection in a PDF, poor topic representation in training data, etc.

Finally, on a similar note, I'd like to take a moment to consider the mechanics of writing. While there are many papers I wish had less reliance on AI, there are nevertheless papers that have sincerely left me thinking "Man, I wish this were written by ChatGPT." I understand that much of academia is built around English, which is a second, third, or fourth+ language for many people I work with. With this understanding, there is nothing wrong with using it to polish---double emphasis on polish---English, using the LLM to ensure the writing is up-to-snuff. What traps people is when they use the bots to generate the English, causing the paper to sound "generated", whatever that means. I don't suggest trying to use tortured phrases, obviously, but a simple paper with little flourish and grammatical correctness will, in my mind, stand on its own much better than any paper whose word diversity matches that of a fourth-grader and whose every paragraph begins with a bold TL;DR. It's a paper. It's meant to be read. I do not care for superfluous length disguised as "reader-friendly" summaries. To be clear, however, I do not mean that all papers should be short. Instead, I believe that all papers should be clear, concise, and correct. If you find this difficult to achieve, good. And, as a related note, please write emails yourself.

Alas, I digress. There are certainly ideas I'm missing, but these points of practice evoke the most emotion from me and they must be digested to start thinking about my arguments to come.

The philosopher's stone

Consider now a hypothetical LLM that can generate anything and everything correctly. While this is not yet real, it must be considered in the limit of powerful LLMs. This brings to mind several salient questions: What exactly is the role of academics in our society? What role does my/your/our research fulfill? What makes a "good" or "bad" paper? What effect does more time have on research? I will not provide any satisfying answer to these questions---much ink has been spilled on these matters---but instead propose practical ideas guided by these questions.

I will not pretend that academia's purpose being misunderstood by the populace or government is a new phenomenon. For example, the Romans' largest impact on the course of mathematics is that they prematurely slaughtered the scientist Archimedes; they made little-to-no use of public resources for abstract mathematics (unlike, say, the Greeks or Syracusans). We currently face an onslaught of similar pressure, not just from contracting funding resources, but from an ever-popular idea that technical institutions serve as convocational degree factories for engineers and white-collar workers.

Personally, I resonate strongly with Plato's statement on mathematics, "God geometrizes continually." In other words, it is not our purpose to use geometry for some specific purpose. Instead, the description of geometry is, in and of itself, the object. To be clear, I do not object to accelerating research into life-saving pharmaceuticals or mission-critical engineered systems. I would bet, though, that you (the reader) are probably not working on these things, and I generally stray away from claiming any research objective of mine as the deciding line between life and death. Instead, I believe our job is to synthesize and impart knowledge (notably different than a degree that dubiously measures such knowledge), which we can only do once we accumulate understanding and intuition. I am no expert at pedagogy, but it seems reasonable to suggest that knowledge is derived from understanding, and understanding must be created internally. Similar for intuition.

There is no substitute for understanding and intuition: You may lead a horse to water, but you cannot make it drink, nor can you make it intuit that you wanted it to drink for that matter. The belief that LLMs can help you understand a thesis's worth of knowledge in a matter of hours or minutes is, in my opinion, demonstrably false and symptomatic of corporate-like pressures on academia. In this sense, we consider the usage of AI as an accelerator for the popular belief that we should replace the abstract search for understanding in research with a more concrete and convocational attitude. These ideas of knowledge and understanding lead me to a few particular suggestions.

It used to be the case that one needed to spend considerable time developing an idea before one could submit a paper, which clearly has consequences. For example, this old pattern discourages negative results, leads to greedy areas of research with incremental (i.e., "hole-filling") topics, and promotes terse papers written hastily. The idea that a paper can be generated near-instantly gives a curious new perspective on these problems. It becomes clear that finances incentivize such hole-filling behavior, as the torrent of such papers has only increased and I similarly still see few negative results. Nevertheless, we now no longer need to have convictions about what ideas are worth our time to pursue. LLMs instead allow us to create papers that are of interest to virtually nobody, including ourselves. Even a handwritten paper based on LLM-generated work that you do not care for will reek of indecision and carelessness; using the model to write the paper only exacerbates such problems. It is our job to care about what we are doing, and believe it or not, such conviction often comes across in great papers. Further, it helps others decide on where to invest their time. If you don't believe in your work, who will? Who should?

On a more concrete level, I advocate strongly against anyone without a masters thesis (or equivalent thereof) using LLMs for research or research tasks. In particular, this includes coding. In applied mathematics, I hold the firm belief that the code/implementation is part of the contribution, even if the research code is not intended to be used outside of the paper. The nuance of many papers lays in the codebase, and there is no shortcut for understanding here as well. For example, it takes nontrivial understanding of finite element methods to know the different implications of doubling the number of elements versus doubling the nodes in each element, even though these change the computational expense identically. Similarly, understanding the practical strengths and pitfalls of, e.g., sequential Monte Carlo, often helps me immensely when evaluating papers in generative modeling. Writing the code is vital when understanding your work in the larger context of research. I will discursively add that finishing a masters does not give one a "free pass" to use an LLM maximally (nor does finishing a Ph.D., postdoc, tenure-review, ...). I include this, though, as a simple condition that I think should be held firm. I do not desire to police each level of academic, but as a guideline, I would meekly suggest that one should have a masters-level understanding of the particular research topic you are using the models for.

Regarding the other part of communication, I cannot overstate how firmly I stand against using LLMs to generate a presentation. On one hand, I recognize the survival bias: I can only know a presentation is generated if it is bad. On the other hand, I have seen even good presenters give terrible presentations when they were constructed using an LLM. A major problem is that I do not think the LLM's training data represents such presentations very well. I doubt it ever can, though perhaps this goes against the "philosopher's stone" concept. Further, constructing a presentation is usually vital to understanding its content. Instead, consider structuring the presentation yourself first, deciding how things should look on paper, perhaps using powerpoint/keynote/slides to construct a rough draft of the presentation's "rhythm." Only then, I think, is it okay to use the LLM specifically to create the presentation you first sketched out.

Other considerations I must make

I now mention extrinsic objections to AI usage, most notably being its threat to labor and the environment. It is unclear at this very moment how these issues will evolve, so even I regard my own thoughts as inchoate.

On the topic of labor: this is something that each individual must reckon with. I do not fault someone for using a model for tasks that reduce the tedium of busywork (forms, annual reports, logistics, etc) or for accelerating and exploring research topics within the aforementioned confines of responsible AI usage. I do, however, fault someone if they use it simply to cooperate and acquiesce to "larger market forces." Surrendering responsibility under the guise of accelerationism makes a weak academic, in my opinion. Should we not be robust to failure? Do you truly believe that academia (which clearly is not industry) is a "market"?

It is widely said that job losses will be incurred across the public and private sector, not to mention the counterfactual of openings that are never realized. I may suggest that you use the AI responsibly for the purpose of enabling yourself without reducing your dependence on others. After all, if an LLM means that one professor can do the work of two, do you think budget-constrained departments will still hire both professors? What mental tax would a doubly-burdened professor bear? Further, academics should be amenable to collaboration (in theory, at least), where we can depend on others' areas of expertise and ask them for advice/references/thoughts when we encounter a problem they may know more about. There is, thus, a social consequence where the usage of LLMs can beget research isolation.

On the topic of environmentalism: I really have no satisfying opinion. Any unbiased information about the models is held deeply confidential, making it difficult to judge the cost of inference or training. Further, the current boom in data centers makes information even more scattered and hard to come by. Nevertheless, I urge you to think about the capitalist viewpoint: It is estimated that Anthropic has 2500 employees. While they are surely paid handsomely, consider that the startup raised $65B dollars in a May 2026 funding round, enough money to pay each employee seven figures annually for the next 25 years. While the engineers paid rather handsomely, I am certain most of this cost is for infrastructure development, and one can do a lot of environmental damage with a budget of that magnitude. How much water will it use? I cannot say. How much energy will the next version of Mythos take to train? It's impossible to know. Nevertheless, the environmental consequences cost hard money to overcome, and the budget must at some point acclimate to the fiscal reality of these costs.

The water must come from somewhere, the energy must come from a power source, the electronics must be made from raw materials, and they are using (tens of) billions of dollars worth of each of them. These thoughts are clearly incoherent, but that is because I firmly believe the environmental aspect of these models is illegible. Subjectively, I find the virtual non-knowledge of simple environmental facts about these companies appalling and I believe that any self-conscious environmentalist would be best served by avoiding the use of AI for more than the "one-off cases" suggested above. I find it difficult, however, to fault one if they think differently, as we simply do not know the truth of the matter.

Finally, I finish the "philosopher's stone" topic with a simple question: How much would a real philosopher's stone cost? While I suggest that academia is not a marketplace above, the AI industry absolutely is. Maybe you are at an institution now that can bear this cost---MIT is fortunately well-off---but what if you move on from graduate school to a resource-strapped local college? Will you pay the 200 USD/month subscription out of your own pocket? Will you be able to attain tenure without it? What if the model you depend on for research increases to 500 USD/month? 1000? While the $65B figure cited above is eye-watering, Anthropic is still nowhere near profitable. These costs will rise. Agents will cease to be profitable at the rate of compute increase we are seeing. Even if a particular company becomes profitable, the way you use a model is not guaranteed to exist in a year. Month-to-month, week-to-week, or even day-to-day changes are clear. Thus, please consider that you must develop every skill necessary to get by on your research if all models you use are shut down tomorrow, if not for the sake of science then for the sake of capitalism. There is empirical research suggesting that the skill of critical thinking deteriorates with the dependence on AI; I skeptically await replication and reproduction of these results across independent trials before accepting this as scientific fact. It is, nevertheless, something to be aware of, and you must consider that your research abilities today may not persist into next year without regular exercise.

Parting thoughts

If I am to avoid preaching to the choir, it might be necessary to write some nice-and-tidy synopsis here for those whose attention is so frayed that they cannot read an essay. I will avoid this temptation. Throughout this, I have asked many accusatory questions of you, the reader. It is not solely because I am rather conservative in my opinion of the new technologies facing us, though this is part of it. It is also because I find many anti-AI thoughts expressed by the general public as incomplete, making it rare that the average researcher (not the AI enthusiast, not the luddite, but those of you who would find something interesting in this essay) reckons with the extent of the issues I believe this technology poses. I acknowledge my own biases here, but that does not detract from the fact that any technology that bends society to its will must beg questions that we simply cannot ignore. So I ask one last, simple question: Are you really, truly comfortable with the reason that you use AI to the extent that you do? If the answer is no, then perhaps change your usage or perhaps change your reason. If the answer is yes, though, congratulations---you are probably happier than me.