Scott Alexander created a Turing Test for AI generated artwork. Begin quote:
Here are fifty pictures. Some of them (not necessarily exactly half) were made by humans; the rest are AI-generated. Please guess which are which. Please don’t download them onto your computer, zoom in, or use tools besides the naked eye. Some hints:
- I’ve tried to balance type of picture / theme , so it won’t be as easy as “everything that looks like digital art is AI”.
- I’ve tried to crop some pictures of both types into unusual shapes, so it won’t be as easy as “everything that’s in DALL-E’s default aspect ratio is AI”.
At the end, it will ask you which picture you’re most confident is human, which picture you’re most confident is AI, and which picture was your favorite – so try to keep track of that throughout the exercise.
All the human pictures are by specific artists who deserve credit (and all the AI pictures are by specific prompters/AI art hobbyists who also deserve credit) but I obviously can’t do that here. I’ll include full attributions on the results post later.
I got 88% correct (44/50). Here’s my attempt (or rather my imperfect memory of my attempt), and my justification for the answers I gave.
Human. Stuff like the cherubs holding her skirts (one blindfolded, one a cyborg) read like the kind of deliberative creative choice that AI never makes. The birds and feathers and hair (typical pain points for AI) are all sharp and coherent.
Human. I haven’t seen an AI-generated image like this before. Tree branches overlap believably (instead of disappearing/multiplying/changing when they occlude each other, as in AI images). 
Human. Has an AI feel, but the hair strands look correct, and the frills hang believably across her shoulders. Image is loaded with intricate, distinct-yet-similar details (waves/foam/fabric/clouds/birds) that never blend into mush.
AI. Blobby oven-mitt hands. Deformed foot. Nobody in the 18th/19th century would allow their daughter to be painted wearing such a scandalous dress.
Human. Coherent and symmetrical to the smallest detail. Note the dividers on the window.
Hard call. I went with AI because of the way the blackness of the woman’s hair abruptly changes intensity when the curving line bisects it. A random, unmotivated choice that a human wouldn’t make, but a machine might.
AI. Right hand has two thumbs.
Human. Haven’t seen any AI images like this before. The sleeping disciples are sprawled in complex but believable ways.
Human. I went back and forth, but decided the red crosshatching pattern was too coherent to be AI.
AI. I recognize the model that created this: Dall-E 3, which has a grainy, harsh, oversaturated look. Other clues are the nonsensical steps to nowhere, the symmetry errors, and the kitsch colors of the gate, which detract from the sense of ancient grandeur. Nobody would spend so much time on the details of the stonework, only to make it look like it was built out of Lego blocks.
A hard one. I guessed AI because of the way the windows/chimneys of the houses appear to be slanted. No reason a human artist would do that. (I didn’t see the nonsensical “signature” in the bottom length corner; if I had, I would have guessed Human.)
Guessed AI, got it wrong. Very hard.
Not hard. At least 10 separate details would make me guess “AI” at 90% confidence. The hairstrands are a melty incoherent mess. The hands have froglike webbing on them. Her skirt is excessively detailed and its metallic designs lack symmetry. Her earring floats in space. It contains confused stylistic choices. The girl’s face is a simplified anime design: does that agree with the ultra-detailed skirt and the near-photorealistic water? Also, where is the light source coming from?
AI. Not sure what made me think a human hadn’t made it. Maybe the lack of doors: how do you get in and out?
Human. Many specific choices, plus the text on the temple.
heavy AI slop feel. Many occlusion errors. Deformed hands. Probably Dall-E 3 again.
I actually can’t remember how I guessed here but it’s pretty clearly AI. The face’s left eye is deformed in a random, unmotivated way. Occlusion errors. Filled with ugly, harsh artifacts. Seems like it was trying to write letters in the middle of the image and gave up.
Human. Mild slop aesthetic (pretty girl + shiny plastic skin + random sparkles/nonsense) plus heterochromia should point to AI, but the hairstrands are too coherent.
Human. Don’t know why I gave this answer. Was probably a guess.
AI. What’s the middle black rectangle in the house? It can’t be a door, because it doesn’t reach the ground. If it’s a window, why doesn’t it match the other two?
lol, get this cancer off my screen.
what makes this look human? I don’t know. Maybe it’s the stark, understated tree? AI would probably put dramatic explosion-esque leaves or flowers on it to match the sky.
A good example of the Midjourney slop aesthetic. It superficially invokes liturgic religious iconography, yet Mary wears a full face of Instathot makeup and has detailed veins on her hands. Very weird, inhuman choices. What’s the little nub of flesh poking between her right thumb and index finger?
Midjourney slop. Occlusion errors on the hairstrands. It tried to give her an earring but didn’t finish the job, leaving a strange malformed icicle hanging off her ear.
Human. This was a bit of a cheat because I’d seen the image before: it’s a render by Michael Stuart and was my desktop background for a while. If I hadn’t seen it, I would still have said “human” based on the complex, coherent rigging and ratlines.
AI. Mangled hands. Man wears two or three belts at once, and the holes don’t make sense. His loincloth tore in a weird way, giving him a Lara Croft-esque thigh holster strap. An interesting example of stylistic blend: it’s going for a Caravaggio-esque painting, but mistakenly puts all sorts of painterly details into the setting itself: notice how the man’s left hand appears to be holding a pen or a brush, and how the windowsill is a gilded painting frame.
AI. Looks like a Midjourney image from 2 years ago.
Blindly guessed “Human” and got lucky.
Human. The windows look coherent.
AI. Got it wrong. I thought “what are the odds that two such similar looking pictures back to back would both be by humans.
AI. The cafe has seemingly hundreds of chairs and tables, some of which are overlapping or inside one another. Why are plants growing inside the window frames?
Guessed AI because of the numerous mistakes in the reflections. It wasn’t. Come on, man.
So sloppy that Oliver Twist is holding up a bowl and asking for more.
Human. I was rushing, and becoming careless. Seems obvious it’s AI in hindsight, when you see how messy the fruit is.
Human. The musculature of the man’s torso is consistent with how human bodies were portrayed by late-medieval artists (not all of them could pilfer corpses), but inconsistent with AI, which is mode-collapsed into anatomical accuracy (see the painting of the Blessed Virgin Mother above).
Human. It’s unlike AI attempts I’ve seen at this style. The signature looks real.
Looks like a 4 year old GAN image.
I think this is by Dall-E 3. It loves excessive pillars.
Human, but I don’t know what makes it human. I haven’t seen AI images like it.
If AI slop was a videogame, this picture would be the final boss.
Messy, but packed with deliberate human choices and intent. Characters interact with each other in complex ways. Branches and trees look correct.
Human. I’d seen this before.
AI. A very old image.
Guessed human. Whoops. It was AI.
AI. Reflection errors in the water. The tree roots on the left look wrong.
More slop. Surface of the landing craft is just random shit (and its feet are asymmetrical). Right astronaut has a weird proboscis sprouting from his helmet.
Human. Complex interactions between positive and negative space. The cutouts are chaotic yet have a congruent internal logic. I’ve never seen an AI image like it. Some of the cutouts have torn white edges—a human error.
Human. Difficult to say. I think AI images generally either have a clear subject or a clear focal point.
AI. Another hard one. What swayed me was the second door on the left-side building. It seems to exist off the edge of the land.
We have such slop to show you. Cute big-eyed robot + staring directly at viewer + meaningless graffiti sprays + meaningless planets and circles to take up space = Midjourney. Concluding Thoughts
You’re in a desert walking along in the sand when all of a sudden you look down, and you see a tortoise, it’s crawling toward you. You reach down, you flip the tortoise over on its back. The tortoise lays on its back, its belly baking in the hot sun, beating its legs, trying to turn itself over, but it can’t, not without your help. But you’re not helping. Why is that?
No Comments »
Comments are moderated and may take up to 24 hours to appear.
No comments yet.