Can I trick Pangram into thinking I'm a bot?
Also: Summery meal ideas lol
Hi Friends,
It’s been a long while since I wrote on Substack. Mostly I’ve been heads down working on my over-edited video about AI. Please enjoy it if you haven’t:
I also did more parenting while my wife did extra work stuff in July. That meant cooking and serving all weekday dinners. At first I figured I’d simplify. Lots of rice and beans. But then I had a memory of growing up when my mom would go to a late meeting and my dad had to cook and he’d make something kinda cool like meatloaf. Only problem is he’d serve it with nothing. No starch. No vegetables. Just meatloaf and ketchup on your plate. It was depressing.
So with that in mind, I decided to overcompensate and used July to make some elaborate meals.
Am I now bragging about that on Substack? Damn right. What are you using your Substack for?
TODAY’S TOPIC: BOTS R US
The latest AI controversy involves a canceled $2 million+ two-book deal. The author, Jerry Falade, claims that he didn’t use AI to write his book. He claims Pangram, the AI-detection tool, regularly marks his writing as AI, even stuff he’d written in 2017.
Falade was born in Nigeria, and there is evidence that these AI detection tools work less well on authors who didn’t learn English as their first language or who grew up speaking an English language variant like Nigerian Pidgin. (I don’t know Falade’s personal language history).
But the white American person on Substack I learned about this piece from mentioned that she too has posts rated by Pangram as 80% AI, when she knows she wrote them completely.
This got me thinking. Could I learn to write like the machine? Surely a nimble-footed, voice-aware writer like moi could fool Pangram into thinking I was a chatbot. I’ve communicated with the chatbots enough to know the cut of their jib. I saw the telltale AI quirks in literary agent Mark Gottlieb’s writing.
How hard could it be?
below is me trying to write like a bot. first attempt:
I set out to do the impossible.
For many months now, we’ve been hearing of the stilted style of AI writing. We’ve seen the telltale signs—the “not X, but Y” formulations. The love of lists with three items. The capable if not thrilling vocabulary.
But could I, a human being named Blake Lefray, successfully mimic the AI enough to trick Pangram, the industry-leading AI detection tool?
There was only one way to find out.
First, I needed to establish some ground rules.
I cannot employ any AI tools in my writing process.
I can feed my writing to Pangram as many times as necessary.
Whatever the end result may reveal—I must face it.
At first, it seemed simple. I just needed to write in that faintly incredulous voice, while making sure to vary my sentence length. Easy enough. Then came the hard part.
In the process of writing in the style of AI, something unexpected happened. I found myself lurching from topic to topic. Could it be that this AI style not only reflects mental fuzziness but creates it?
While scientists aren’t able to answer this question today, the future is bright with possibility. After all, if we aren’t the ones to imitate the AI—who will?
god damn it!
100% really?
whatever.
time to try again.
I imagine the author, Blake Lefray, was surprised by the results. The truth is he has an outdated view of AI writing ticks. Let’s explore where he went wrong.
1. The idea that generative AI writes in a “whiny” voice.
This was true not long ago. But AI has made meaningful strides in the past several months as developers have tweaked the models to avoid this tone. Contemporary models write in a less exaggerated tone. The style could be described as “matter-of-fact with a little edge.”
Lefray’s attempt to satirize this tone likely led to the AI detection tool seeing it for what it was. A fabrication.
2. The dramatic register of AI writing
LLM-generated writing is known for making bold statements. The models have been trained to write in a style that captivates readers, and readers tend to prefer bold statements over bland ones. But Lefray takes it too far.
Take this sentence—“I set out to do the impossible.” Lefray has taken a meaningful observation about LLM’s favoring dramatic lines, but he’s made it a cartoon. The distinction is important.
Today’s LLMs, while unable to perfectly match the cadence and humor of the sharpest human writing, are no longer the clunky writers Lefray seems to imagine.
The real question is this. What happens when the bots get even better?
what the hell dude. why do i suck at ai writing so much?
i can’t go on. i’m going to break my own rule. since pangram isn’t telling me what i did wrong, i’m going to ask chatgpt why that portion wasn’t fooling pangram.
*i feed it to chatgpt*
it says my writing isn’t generic enough.
ha. you want generic? you get generic.
AI writing often lacks an argumentative arc.
Paragraphs are the building blocks of essays, and sentences are the building blocks of paragraphs. But to make an essay great, it must have a through-line.
A human author may attempt to write without a through-line. It’s harder than it looks.
Pangram is the industry-leading AI detection tool, but it can make mistakes. Those mistakes can have devastating consequences for the parties involved.
Call it a lack of trust. What happens when we can no longer trust the authors and publishing industry?
These questions are easy to pose and hard to answer.
i am a failure.
i am no true writer. a real writer would be able to imitate the ai.
or maybe the tool was a bit off. frustrated, angry, i ask chatgpt to generate a follow up. i want to see what pangram will say. here’s what the bot spits out:
Lefray became convinced he knew what AI sounded like.
He made the sentences flatter. He trimmed the adjectives. He added the occasional bold declaration. He removed anything too strange or personal. Then he fed the essay into Pangram.
Human.
He tried again.
This time he made the voice more confident. More analytical. He replaced jokes with observations. He ended with a rhetorical question.
Human.
After weeks of revisions, Lefray had a realization.
He wasn’t failing because he misunderstood AI. He was failing because he couldn’t stop being himself. Every attempt to imitate a machine became another document of a human trying very hard to disappear.
Pangram, it seemed, was better at recognizing authors than algorithms.
There is absolutely something super fucking creepy about AI writing that I totally failed to capture in my imitations. Reading what the AI wrote above kinda makes me wanna puke. My attempts were just as boring, but none of them had that faint puke-inducing factor.
I guess I’ll leave you with this: Don’t you think that—excepting non-native English speakers—admitting that your writing frequently gets mistaken for AI writing is kind of a self-own?
Here’s another: Can you beat my feeble attempts? Can you fool the chatbot detection tool into thinking you’re a bot? If so, how’d you do it?
Lastly, August is tomato season in my neck of the woods. Don’t let it pass you by!
Cheers,
Blake
P.S. - As you may know Substack recently partnered with Pangram to allow writers to scan their own work for AI-ness. It failed to notice the AI-generated text in this post. Not a great look for Pangram’s reputation.
As a person over 40, I cannot read any of the yellow text.
Also, your meal looks amazing and I welcome all meal photos you choose to post.
I am constantly reminded how much I do and should hate AI