Don't Use AI for Writing, But What About Research and Editing?

I often read: "Don't use AI for writing. Use it for research." Or: "Don't use it for writing, only for editing." Neither is good advice.

A Convenience or a Trap?

Ever since people started using generative AI for writing, there's been a debate about whether that's a wise thing to do. Now that AI slop is so obvious, the majority says no. For some reason, the same majority seems to think using AI to assist the writing process is fine. But it may cause more damage than most people realize.

Using AI for Research

Using AI for research is handy, but it's also risky. It sets a frame for you and directs your attention in ways you may not want. Sometimes, the way AI presents information is arranged to nudge your opinion toward a single point of view.

LLMs are biased beyond belief, and there are several reasons for that. The most obvious one is that those models are trained on biased material. Next, during the fine-tuning phase, LLM outputs are rated and corrected by biased humans. (Not that they choose particularly biased individuals to train AI. I'm talking about human nature. We are all biased.) There are a few more known sources of AI bias, and you can find them listed in this SAP article. Then there's also a deliberate effort to make AI use politically correct language, which is also biased.

A 2024 study published in Nature Human Behaviour found that when people interact with biased AI systems, they gradually internalize those biases, and they aren't even aware of it happening. The amplification effect was significantly stronger than what researchers observed in human-to-human interactions. It turns out AI not only reflects bias, but also teaches it. The consequences of AI bias are more far-reaching and dangerous than those of us humans being biased.

AI also hallucinates. That's mostly because the training process discourages uncertainty and prefers plausible, inaccurate statements. LLMs are trained to make things up when they don't have the correct answer. The facts they provide often turn out not to be facts at all.

The factual errors are actually the easy part to fix. What causes lasting damage is the bias and the frame that is presented to you. The frame in which you now think about the topic.

Does it really matter that you wrote a piece yourself if an algorithm directed your processing of the topic? Would such a piece be original, refreshing, based on original thought, and written from a genuinely interesting angle? Probably not. So why do it then?

Editing With AI

No doubt AI "knows" all grammar and spelling rules, all style conventions and guides, from CMOS to the more obscure ones, and can mirror the prose style of popular authors from any era. It can absolutely polish your prose. But think about this for a second.

Imagine spending hours, days, or months restlessly working on your writing project. You have something to say and know how you want to say it. You write it yourself. To make it even better, you paste it, piece by piece, into Claude or ChatGPT, just to polish your writing. The output looks flawless, and you adopt it. And the next thing you know, it is flagged for AI content. Why?

The answer has to do with how LLMs actually work. An LLM predicts the most likely next word or syntagm in a sequence, based on what was the most common across its training data (the entire internet) and what responses got the highest ratings during training. Any output is shaped by what an average internet user is most likely to say in any given context and it is most heavily influenced by average AI trainers' taste and preferences. When it comes to "higher" forms of writing, LLMs reflect the style of best-selling authors whom everyone else is mimicking (with or without AI assistance), so again you can't expect anything authentic.

The result is generic because it's optimized for the statistical average of everyone else's writing. Its style sits at the center of all styles. When an LLM edits a text, it pulls your writing toward that center. The more thorough the editing, the less it sounds like you. Your voice gets lost and replaced by a generic one that we all know very well.

That's the first problem with AI editing. The second one can do even more lasting damage.

Whenever we interact with AI, our conversations are recorded and used for improving the LLM. So far so good. But what does that mean? First, if you feed an AI with your unpublished manuscript, it becomes training material, basically a template. It may influence other people's AI-generated work before your own is published. Then, one day, your original writing could be flagged as AI-generated or even plagiarized.

I wrote recently that AI detectors aren't particularly reliable, but that doesn't matter; everybody's using them - including publishers, agents, and self-publishing platforms. Just imagine the consequences.

And that's still not all. The interactions with AI are not just absorbed into training data. They may also be read and scrutinized by individuals working on RLHF (reinforcement learning from human feedback). People actually read your prompts and the generated responses. Not all, but you never know which ones. Your unpublished manuscript is confidential. Treat it that way.

So, Is Using AI for Writing Really That Bad?

No, at least in my opinion. Not everything that needs to be written is equally important. Not everything has a purpose to be authentic. There are things I'm glad I don't have to write by myself ever again.

I am not arguing that you should avoid AI at all costs. All I'm saying is - if a particular writing project is important to you, don't let AI direct your thoughts on it, and don't hand it over as training material. It may become training data one day, but make sure it's copyrighted first.

And there's one more dimension worth considering. When you publish writing that was shaped by AI, whether in research, framing, or editing, you're passing that influence on to your readers. They came for your perspective, your way of understanding. If you let an algorithm direct your perspective, it shapes your readers' views as well. Your "AI-assisted" writing becomes a channel of influence that you don't even control, and it may be heavily misleading. That's worth taking seriously.