People can’t tell when a poem is written by AI

People often praise poets for their soulful turns of phrase and how they can evoke deep emotions in ways that feel uniquely human. It turns out AI might be even better at tugging at our heartstrings, according to a newly published study in Scientific Reports. Not only do people struggle to tell the difference between AI-generated and human-written poetry, but many prefer AI-crafted poems to those produced by human effort, at least until they discover the silicon soul behind the words.

The University of Pittsburgh researchers tested how well readers could identify when a poem was written by OpenAI’s ChatGPT-3.5 AI model or by Shakespeare, Emily Dickinson, T.S. Eliot, and others in the pantheon of English-language poetry. Over 1,600 participants read a random mix of ten poems, half by humans and half by the AI model. Not only did many think humans wrote the AI poems, but the poems written by people were least likely to be marked as such.

Apparently, the complexity of human poetry was mistaken for confusing AI rambling. By avoiding the complexity often found in the work of classic poets, AI poetry can feel more relatable and less intimidating – qualities that readers unconsciously attribute to human creativity.

“We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored,” the researchers wrote. “Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.”

Poetic AI

The inability of many people to tell when a peom is written by AI is surprising, as is the preference for poetry written by AI until the author is revealed. But it’s more a sign that poetry isn’t always easy to parse, especially when it’s not contemporary. And it means AI is slanted toward appealing to the most people possible today, just like it’s other output.

We often assume that human creativity is inherently superior or that we can intuitively recognize the work of a fellow human being. Yet, as AI tools improve, those assumptions are increasingly put to the test. This isn’t just an academic exercise, either. It has real implications for art, education, and how we value creative work in a world where machines are now serious contenders.

The findings also suggest that as AI becomes an increasingly sophisticated creative tool, we may need to rethink traditional definitions of artistry. It’s not necessarily about whether an AI can “feel” or “imagine” but about how its output resonates with the audience.

But, perhaps it’s best to leave the last word about being human and poetry to a poet who wrote a lot about both. Here’s “I Am a Parcel of Vain Strivings Tied” by Henry David Thoreau:

I am a parcel of vain strivings tied

By a chance bond together,

Dangling this way and that, their links

Were made so loose and wide,

Methinks,

For milder weather.

A bunch of violets without their roots,

And sorrel intermixed,

Encircled by a wisp of straw

Once coiled about their shoots,

The law

By which I’m fixed.

A nosegay which Time clutched from out

Those fair Elysian fields,

With weeds and broken stems, in haste,

Doth make the rabble rout

That waste

The day he yields.

And here I bloom for a short hour unseen,

Drinking my juices up,

With no root in the land

To keep my branches green,

But stand

In a bare cup.

Some tender buds were left upon my stem

In mimicry of life,

But ah! the children will not know,

Till time has withered them,

The woe

With which they’re rife.

But now I see I was not plucked for naught,

And after in life’s vase

Of glass set while I might survive,

But by a kind hand brought

Alive

To a strange place.

That stock thus thinned will soon redeem its hours,

And by another year,

Such as God knows, with freer air,

More fruits and fairer flowers

Will bear,

While I droop here.

You might also like