AI Detectors Aren't the Fix. We Need New Norms Around Author Disclosure

By the end of this essay, you’ll use A Cow to place yourself on a scale. Only you will know where you end up. There’s no button or program that can tell you. No gray-and-green percentage bar to declare your fate. Before that, though, we have to find a common language. We have to stop being distracted by the false targets driving the AI-and-writing conversation. It’ll take some honesty. So, don’t cheat before answering the following:

What’s your definition of “AI writing?”

Did you come up with something coherent? Complete? Could it land on a syllabus, a corporate policy, or a legal brief? Or did you invoke the spirit of Justice Stewart and think something along the lines of, “I know it when I see it?”

This week, Substack rolled out its embedded AI detection software to much fanfare, consternation, and grinding of teeth (Gah! A list of three!). It’s far from the first. Industries are and will continue being built on the detection of AI-generated material. Towering, gray, faceless structures to peer over your shoulders and whisper, “that’s pretty suss.” Generative AI is not going away and neither will the tools to detect it. Because if you can’t solve the problem you’ve created, then you might as well profit off of it.

The AI detectors themselves are a red herring. They aren’t the core issue and they aren’t the best solution. In the future, they might play a role. Once polished up and refined. But for now, arguing over them is like fighting over wagon frames before the wheel has been invented.

Writers lack a common framework on which to build new norms.

Nobody needs another paragraph about AI’s impact on writing, and this won’t be one. The cow I propose is not a Golden Calf from which morality and commandments arise. The tool is descriptive, not prescriptive.

Why focus descriptive when it comes to AI writing? Because a profound need for new understanding has been paired with unclear communication about it. That combination is one part headache and two parts collateral damage.

Pick any article or post about AI right now and read the comments. It’s chaos. Like competing towers of Babel, people are constructing their monuments to self-righteousness all the while using words and terms that nobody else knows the meaning of.

The only reason an AI-detector is a plausible “need” is because there’s a lack of transparency. Will there always be bad actors who attempt to deceive others about their use of this technology or that? Yes, definitely. And those people deserve the void.

But the problem facing every normal, reasonable, generally-well-meaning writer out there is this:

The discourse is so choppy and antagonistic, so full of all-or-nothing accusations, that even rational people are disincentivized to talk.

Not only does this kill conversation, it stifles the kind of conversation that could lead to understanding and new norms. AI-writing becomes a catch-all term and a scarlet letter such that writers of all kind are incentivized to hide any and all use. The detectors, like the one now trolloping across your Substack feed, attempt (however imperfectly) to capture only a sliver of what a concept like “AI writing” contains. They also miss the point.

You might think that fine tuning what “AI writing” means isn’t the main issue. I agree, it’s probably not the main issue - but ironing it out is a prerequisite to every other conversation, and yet it keeps getting skipped over.

Detectors are not the solution. They are a tool, but they are woefully inappropriate tools for the core issue. The truth is that self-identification about AI use from writers and by writers was always a necessary condition for figuring out, as a larger community, what is acceptable and what is not. Think back to high school, and the very basic lessons on plagiarism (they have detectors for that, too, you know.) Good teachers did not stand on the desk and hurl dry erase markers at any student who would dare incorporate another’s ideas into their work, while encouraging the rest of the class to join in. Instead they defined what counts as acceptable or not. What defines attribution, or acknowledgement. What the rules are. None of that comes from a vacuum - it’s convention that took time to evolve. Given the current talk about AI-writing, do you see that kind of discussion anywhere? It’s hard to find.

Progress won’t happen without understanding, and understanding won’t happen without nuance. Things are tribal, and generalized, and ill-defined. The conversation is stuck because we can’t even agree on what we’re talking about. I cannot imagine things getting better until we fix that.

Failure to understand another’s position almost always underpins poor conversation. I do not believe there will be a clear, bright line when it comes to AI and writing. It will evolve, it will be fluid, it will be a tapestry of standards. An ethic with an irregular edge will always require granularity and common language.

It needs A Cow.

AI Detectors Focus on Product. We Should Care About Process.

Right now, do the impossible and forget everything else you think about the AI detector. Every conversation about AI detectors smuggles in assumptions that need to be unpacked. Don’t waste energy worrying or fighting about whether the skyscraper has an ugly spire when the foundation is primed to crumble.

What my proposed bovine mechanism for AI identification does share with the detector is an impulse toward transparency. This is a relatively new problem for writing! Sure, there used to be problems of plagiarism, rip off, imitation, and parody. But the writing itself was an assured thing. Somebody, some human being, had to sit down and put down the words.

That’s gone. Forever. Which means that for the first time in history, writing faces the same questions that free-range eggs, blood diamonds, and Fair Trade coffee have had to grapple with: what process produced this product?

If a write states that he or she "accepts all the help she can get" and uses "modern tools," what the hell does that statement mean? “Accept all the help.” Grant yourself an imaginary world where you’re confident the detector is accurate It’s all fixed. Hooray. If that ideal AI scanner comes back saying a piece is “100% human written,” do we have any idea about Joyce’s process as a writer? What if it’s 50%. Is that really the sum of all her “help?” Do you even know what she means?

I don’t.

That’s the problem. The rotten root under the soil that is poisoning the vegetable garden. There’s no understanding of what “AI help” let alone “AI writing” means, and no position from which to talk about it.

Transparency is needed. I can see no excuse to not label AI involvement in writing. However, even if the AI-detectors had all of their flaws stripped and all of their implementation perfected, it would still be a flawed paradigm. An incomplete picture. We have to step back. We have to open a more patient, curious, and calm conversation. We can only do that with common language.

This Isn’t an Original Idea

Unsurprisingly, there are smarter people than I thinking about this same issue.

At the most basic layer, various publications, ethicists, and organizations have some kind of loosely-defined scale in the “On AI” tab of their webpage.

More often than not, I see them break it into a vibes-based scale, where 1 is “No AI Used” and 5 is “It’s Pretty Much All AI.” Great for a gestalt but (in my humble opinion) they all share the same fundamental problem:

They look retroactively at the end product, not proactively at the process.

Writing, reporting, storytelling, and every other creative word-based project is not some Lego replica of the space shuttle, where a careful accounting can tell me exactly how many bricks were placed by you, and how many the Lego Assembler 3000’s claw laid down. Ideas propagate, they build, they branch out. If you wrote essentially everything but the detailed outline came from AI, is that a 1 on the 5-point scale? It doesn’t feel like it.

Within education, there’s some popularity to the “AI Assessment Scale,” from Perkins, Furze, Roe, and MacVaugh. It’s much more robust than your typical instructions sheet. If I were a student reading the description I might have clarifying questions, but I’d feel much more confident in what is being asked of me.

Critically, their paradigm turns the focus onto process rather than product. It’s more focused on education and scholarship, but you can immediately see parallels to writing and other creative endeavors.

You can find AI scales circulating in corporate worlds and other sectors, including computer programming. There’s overlap, sure, but we need something specific to writers and writing. Broad applicability is too often the enemy of utility.

I Propose: A COW

Not “a particular cow.” The A COW Scale.

The AI Contributions to the Output of Writers Scale. Version 1.0.

The A is part of the acronym which means a lot of the above mentions have been bastardizations (AI would never.) Version 1.0 because we have to stay humble.

Like good games and bad parties, this scale has rules. Constraints, assumptions, and limitations.If you want to skip to the rules, please do so. But ignorance on the terms and conditions is not a platform from which to disagree.

The Rules:

1. It’s About Writing: Writers need to have the conversation when it comes to writing. Yes - agents, editors, publicists, and everybody else part of the writing ecosystem should take part. The consumers (readers) will have a voice on the other end of things, when they speak with their reading selection. But they can only voice that once there’s a way to know what they’re buying.

2. It’s About the Writer: The scale is specific to you as a writer. If you work in finance for your day job and ChatGPT writes your emails then ignore that for these purposes. That’s a separate conversation.

3. Honest Self Reporting: You have to be honest. Examine your thoughts. Do you feel like you’re squinting to land at a certain level, or avoid another? Do words like “well” or “but” or “just” populate your internal monologue when categorizing yourself? Come on. Be honest with yourself and with everyone else. I am no fan of the “AI Witch Hunts” and the attempts at cancelling others based on speculation. People who lie about their use, however, deserve to be lambasted.

4. A Scale of Type, Not Degree: “Look,” I hear somebody say. “AI wrote one paragraph for me. One. That OTHER writer used it to write pages worth! We shouldn’t both be sixes! (Spoiler on the scale)” I hear you. Granular scales of “robot versus fleshy fingers” exist. That’s basically what Substack has! It’s not the point of this scale. You’re both in the same category. This same complaint can apply to “I asked one question of AI” versus “it produced a ten-page historical outline.” There are yet-more-fine-tuned discussions yet to come, I’m sure. That is not this Cow.

5. It’s an Ordinal Scale: Which means that level 5 is more AI involvement than level 2, even if you didn’t specifically use AI for the things in level 2 (though it’s likely you did.) Ordinal also means that the distance between 3 and 4 isn’t necessarily the same distance between 6 and 7. Phew. That’s enough math.

6. Modern AI Exclusivity: This scale applies specifically to modern generative artificial intelligence - the common combination products of AI agents and large language models. If it resembles Chat GPT, Gemini, Claude, or Grok then it counts. The fact that Clippy used to edit your Word documents is not a meta-argument about the continuous, progressive ubiquity of AI in everyone’s work.

7. The Principle of Permanent Touch: AI involvement at any point in the creative process for a particular work defines AI involvement for the entire work. That is to say, if AI is involved in early parts of the writing process, it carries forward. There is no sense in which “I only used AI in the first draft” absolves the final draft. Writing is a fundamentally continuous project. Did you insert a paragraph of AI generated text in the first draft, but it got trimmed or heavily edited by draft ten? Still level 6 (or higher.) I see no defensible position in which inputs to earlier versions can be considered separate from the final. Fight me on it.

8. Meta-Ethical Agnosticism: There are important, difficult conversations around the core ethics of modern AI systems. These include, but are not limited to, training data, copyright, environmental impact, and existential risk. Big ideas that need wrangling. This cow is not running that rodeo. We don’t even have descriptive language - that’s what we’re addressing here. This scale puts pragmatism in front of idealism.

9. Everything is Fluid: If somebody sounds totally certain about all this AI stuff, then don’t trust them. My ideas, this scale included, are in constant flux. I reserve the right to change my mind or restructure my thoughts based on time, new information, and great input from others. Once again, it’s a tool to start figuring problems out, not declare them solved.

Gaze Into A Cow’s Eyes and Find Yourself (on the Scale)

Without further rambling, this is the scale. One to nine in increasing order of AI involvement. Are some of these labels not real words? Hmm. Fascinating.

1. The Purists: Purists reject the use of AI in their writing in any and all aspects, so far as they are capable of eliminating them. Their individual creative writing process would remain completely unchanged were the technology to vanish tomorrow. They live in the same AI-filled-world as everyone else, but they turn off optional tech and actively avoid writing-related interface with any AI.

The Purists might say things like:
- “Any and all AI use in writing is unacceptable.”
- “If you can’t do it with a pen and paper, then it’s not real writing.”
- “It doesn’t matter what you ask AI about - it will affect your writing.”

2. The Orthogonalists: Orthogonalists utilize AI to answer general knowledge questions which are not project-specific, and are non-contributory to the creative content. Instead, they ask questions which inform the writer’s greater understanding of writing itself, or the ecosystem it exists in.

Examples include: Asking AI for definitions of writing-related topics (Ex: “What is a query letter?” “Give me a list of the sub-genres within science fiction”), clarifying formatting practices and conventions, and generalizable, technical-level topics (Ex: “Do you always need a line break for dialogue?” “What are the POV options?”)

In theory, all AI involvement could occur either before a creative project has even been imagined, or after the completion of said project. The specific AI’s involvement could apply in the exact same way to multiple, unrelated projects. Asking if quotes go before or after periods is common craft knowledge.

The Orthogonalists might say things like:
- “I’m only using AI for things people would find on an FAQ page.”
- “My writing is completely my own, but I have no idea how the writing world works.”
- “For me, AI is like a textbook in a writing class.”

3. The Utilitarians: Utilitarians use AI in ways which are project-specific, and indirectly affect the creative output via knowledge rather than craft. Examples include researching a specific topic (Ex: “What kind of bio-signatures would be evidence of life on another planet?”), fact checking, summarizing primary resources, and defining concepts (Ex: “Are there any theories sociology about how people behave in sporting events?”)

[Pause here for a second: Do you feel “the itch?” The urge to say “well, the way I use fact checking is basically what the Orthogonalists (2) do.” Keep reading but take note of that reaction. It’ll happen again. Stay honest with yourself.]

Both Orthogonalists (2) and Utilitarians (3) use AI to acquire general knowledge, but unless your request for “how do semicolons work?” is to inform an experimental punctuation-based literary work, it’s not the same. Utilitarians (3) gain pieces of knowledge that are not themselves creativity, but are scaffolding beams on which creative writing sits. The answer about alien bio-signatures will plausibly affect the direction of your story. Asking “What is a bracket” will not. Finally, while “Common WW2 battle formations” might apply to three of your novellas, but it’s not applicable to all of writing - it still belongs here.

The Utilitarians might say things like:
- ‘“My use of AI is no different than going down a Wikipedia rabbit hole.”
- “It’s a faster way to get information. I always fact check what it tells me.”
- “AI is teaching my things to know, not telling me how to write.”

4. The Brainstormers: Brainstormers use AI to generate, manipulate, or refine creative concepts and ideas that later become writing. They do so without utilizing AI for actual writing of words. AI involvement may not start out as project specific, but the output of brainstorming does ultimately sprout a specific project (that’s simply true of all brainstorming.) The ideas are ultimately filtered through a human’s brain and hands.

Examples include generating character traits (Ex: “What are some interesting professions that would set up for a tense romance?”), plot ideas (Ex: “My protagonist is stuck on an island but I don’t know what he could do from here”), concept workshopping (Ex: “My scene has no tension - they’re just sitting in a car”), and many, many more examples.

Remember - this scale is one of writing process, not final product. It reflects the actual reality, not plausible counterfactuals. Might the idea you end up using be one that you probably would have come up with yourself, or one that you did come up with but only after conversing with the AI? Maybe. Yet it counts here. Hypothetically you might not have needed AI, but in actuality the idea came from or was indirectly sparked by AI.

The Brainstormers might say things like:
- “Using AI is no different than tossing ideas around with another person.”
- “The writing is all mine, I just need some help to get the juices flowing.”
- “All writers get ideas and inspiration from media, life, and random places, anyway.”

5. The Collaborators: Collaborators have text in their work (perhaps earlier versions - see rule #7) that originated from AI. However, it is limited to edits. AI input is exclusively edits of or changes to originally-human writing. This involves entering your writing into an AI in some form.

Collaborators may prefer to call themselves “people for whom AI is their editor.” Because of the ways in which AI works, I don’t think that’s necessarily fair, but the vibe is similar.

The human may end up with knowledge which could have been obtained and applied in a way consistent with the Utilitarians (3), however at this level the AI picked up the proverbial pen and participated in the writing. The process is different, and thus is the involvement.

Edits may be small things (Ex: A run-on sentence is fixed. A quotation mark is adjusted), may be medium (Ex: A continuity error is discovered and flagged), or may be major (Ex: AI tells you a certain scene seems confusing, or the imagery isn’t landing.)

These larger-scope edits may begin to feel like more than mere edits. Yet another reason why “Collaborator” (5) is a better word. If you are getting feedback on a scene, you’ve drifted into functioning as a Brainstormer (4), which is lower on the scale. Compared to the next step, though, the AI is suggesting things without writing them independently. Its writing is only atop what a human had already written.

The Collaborators might say things like:
- “I do the writing, but the AI helps me refine it. The core of the writing is still mine.”
- “The AI is mostly like a better version of spellcheck.”
- “AI is my alpha and/or beta reader.”
- “The AI is mostly my copy editor, and sometimes my line editor.”

6. The Co-Authors: Co-Authors actively outsource some of the creative writing work to AI. The AI generates writing which is included outright in the project, or edited and then included (still including in earlier drafts!)

I suspect that the transition from Collaborator (5) to Co-Author (6) is the hardest to parse. It requires the most individual honesty and reflection. I suspect it will be one of the bloodier battles in the war over AI. I’d suggest considering the following: Many of the Brainstormers (4) end up with ideas that they probably would have come to on their own, given the time and consideration. But in terms of process, they did not. The Collaborators (5) get technical edits, adjustment to pacing and flow, suggestions on weak points in their prose, etc. In theory, they could have done that themselves. Perhaps with time, perhaps with honing their craft in other places. It is the sense that we could, in fact, turn in a perfect manuscript if only we had the energy and space to perfect it. Practically? Maybe not. But there’s no break in it being possible.

This is not so with the Co-Authors (6). They bring into their project writing which originates from another “mind.” AIs voice is not your voice. Its word choice is not yours. The past and personality and experience that you draw your writing from is not what the AI draws from. Vice versa to all of that. The Collaborators (5) include writing that they could have come up with, but did not. The Co-Authors (6) include writing that they would not have come up with.

The Co-Authors might say things like:
- “The AI writes things for me, but then I change it so it sounds like my voice.”
- “When I can’t get something to sound the way I want, I get the AI to give me the line.”
- “When I get to the really boring parts of the writing, I just have AI do it for me.”

7. The Directors: Here, the balance of “writing heavy lifting” shifts. Where the AI is generating the bulk of the writing output with direction from the human. Humans may offer some or all of the tone, vibe, characters, major plot points, themes, and direction, but the connective writing tissue comes heavily or entirely from the AI. The human may have an image or picture of the product in their mind, but the AI is what executes on those ideas. Put another way, the human can imagine the quilt, but the AI is doing the sewing.

Important inclusions: Humans who write a “very rough draft” and insert it into an AI for refinement and production are in this category. In this situation, that draft is functioning as a complex and long prompt, akin to sketching pen on a napkin and asking the AI to turn it into a Monet-like watercolor. Humans who have AI write a first draft, and then edit it themselves are also here.

Practically, a human acting as a director will go through frequent rounds of “yes to this part, but change that section” with the AI. They will rapidly alter and re-generate scenes, chapters, or other large-scale chunks of writing. They themselves may write original material which is included in the project, but it would be a dramatically different (and more difficult) project were they to do it alone.

The Directors might say things like:
- “I’m more of a story teller than a writer. I give the AI my vision and it brings it to life.”
- “The beats and big concepts are all mine. AI executes what I tell it to do.”
- “I’m the one calling the shots and steering the ship, but I’m not writing it down.”

8. The Automators: Automators embrace and seek out the replacement of human involvement in most portions of the creative process. They offer minimal guidance to trigger AI-generated works (Ex: “Generate me a 67,000-word domestic thriller novel that will appeal to 35 to 55-year-old suburban women, with elements of …”). Human intervention, edits, and redirects are minimal, and grossly non-contributory to the final product. A reader (not a writer) could feed an AI instructions for a story they want and get something similar. In theory, another AI that is well-trained and specific could replace human involvement, giving instruction to the “writing AI” or editing along the way.

The Automators might say things like:
- “This is my 40th novel this year.”
- “Oh, is [character name] in this book?”
- “I loved the ending of this book – I was on pins and needles until I read it.”

9. Post-Singularity AI Writers: It’s not on the graphic because this isn’t you. It’s here for completion…and to appeal to the future AGI overlords. They are an as-of-yet hypothetical level in which AI agents have achieved general intelligence such that they create writing works which are 1) original, 2) spontaneous, 3) unprompted, 4) unperceived by humans until read.

The Post-Singularity AI Writers might say things like:
- “Any and all human involvement in writing is unacceptable.”
- “01101110 01100101 01110111 00101100 00100000 01100110 01101100 01100101 01110011 01101000”

Intellectual and Creative Honesty Matter.

No matter where you land on the scale, there are better and worse ways to do it. Lying, like I’ve said, is an unacceptable way. Anybody reading the scale can sense that each category has gradations. The granular discussion about what is and isn’t OK is ever-evolving and much harder to talk about than what I’ve put forward here.

When AI first became popular, I felt more comfortable using it for limited brainstorming than I currently do. My opinions and personal comfort have changed, and I expect every person will go through lots of “versions” of their own preference.

Leave aside all the considerations around publishing, disclosure, popular reading, commerce, and every other capital-O organization. Each of us as a writer owes it to our “writing self” to examine this issue. To think about where we are and where we ought to be. Before we can figure out what direction to go in, we have to locate ourselves on the map.

I Just Need to Know What You Mean.

This will tie back in, bear with me.

I have a young daughter. Watching her learn how to talk is one of the greatest joys of my life. Her language is a window into her mind. That window is getting more and more open, and her thoughts often pour out with no filter. With every new concept she learns, she and I get to discuss and experience more of the world together. I get to learn what she thinks about things, where she’s confused, and what idioms or conventions I’ve taken for granted.

She thinks money and honey are the same thing. One is much more concrete, and I don’t know any way to explain money to a toddler. So when we tell her we have to go to work to make money, she explains we actually already have some in the pantry.

Even non-honey-related concepts get sticky. It took her so long to understand what “first” or “before” meant, because “temporal order” is not a simple concept. If I tried to tell her that we would read a book but first we needed to change her diaper, it turned into yelling and confusion and tears. Eventually she’d start to cry, too.

Words matter. Every writer knows it. If there’s no shared understanding then there’s nowhere for conversation to start. Once there’s a shared language – even just the start – we can start talking to each other like grown-ups.

Thanks for Reading.

Hopefully you don’t feel judged (well, maybe the Automators do just a bit). I don’t intend for this to reflect judgement, even as I have personal thoughts and opinions on what’s “more right” and “more wrong.”

Where are you on the scale as a writer? Where do you think “acceptable” stops?

Did I get it wrong? Counter examples? Is there a category missing? Let me know. Just make sure to be specific in what you mean. Words have meaning, after all. 

Link to The AI Assessment Scale: https://aiassessmentscale.com