I agree with you (and your son) so much that tragically I think I will have to write about it, because this category of error is one that I think many people I adore are prone to. I think there are even Deutschian explanations for *why* it's an error; I think often of his interview with Sam Harris in which he discussed the fact that most people quit addictive drugs because, in a word, they get bored.
Harris presses him on whether a "perfect drug" could be invented that would enslave people to hedonic satiation forever, and Deutsch is emphatic that it could not, for similar reasons to the ones y'all adduce above.
Anyway: fantastic piece, so exciting to read you on this stuff!!!
Nice, I'm glad you arrived at this point -- unsurprisingly it's what I started thinking about after part 2 and I was tweeting back when diffusion models blew up about a civilization-wide taste challenge. You know this is a subject I've dwelled on a lot over the years; I have long been interested in how the candy-coated mind parasites work, stick them in my own brain occasionally to test them, know how to fall back out, etc. I agree that you have the right call to battle, but I think you and James are being a little optimistic. Maybe that's just optimistic battle call rhetoric -- remember art, lads! Art is about humans... novelty is not JUST permutation! -- but I think we're still understanding HOW bad this is.
It's easy to look at attempts to harvest profit on mass taste -- the Beatles, church, Fortnite, everything Disney/Marvel puts out, etc -- and feel reassured that these things move in waves, people get bored of them, there's a counter-reaction, etc.
But your reaction here is exactly, precisely wrong in that "you are looking in the wrong direction, possibly at the past" kind of way:
"itβs really hard to fake SUPER POWERFULLY ENTERTAINING THING, like the show in infinite jest or the joke so funny it kills from the Monty Python bit"
I think we've seen by now that the significant difference with entertainment powered by machine learning is that it's hyper-tailored in the way blockbuster culture-bulldozers are not -- it fills in every other niche and crack in the same way that fan fiction is tailored by communities to fill in every possible crack not covered by a blockbuster (including "this isn't smart enough," I just saw HPMOR out on audiobook). But fan fiction is, by comparison, time-consuming to make, find (especially) and consume, and it works for people who really like to read.
So the taste challenge this time is more akin to an immune system having to resist bioweapons that tailor themselves to individual human cell receptors and can constantly mutate to find more statistical novelty in bounded possibility space. (The COVID pandemic furnishes commonly understandable metaphors and I'm surprised to find don't feel all that remote?) To put it another way, people get entranced by the dream-like quality of generative AI because it's a closer reflection of their own consciousness, individualized culture rather than mass, not a communication with an artist but with your own novelty/pleasure/mental-stimulation circuits.
If I'm going to compare my own foray into this particular taste vortex to things like falling deeply into a few different traditional game genres, into microtransaction-driving sim games, or trying to read several whole decades of comics, etc -- I find the consciousness-feedback-loop with this one much more engrossing, to a degree that's disturbing.
We already know people hypnotized by this effect to various extents, so the question is how long it lasts -- is "get high on a loop with your own subconscious" as dangerous as schlocky 90s sci-fi dreamed it would be, or will it wash over like another culture fad? It's likely to be variable, I guess? I see a lot of people stretching aesthetic muscles and trying to articulate for themselves and others what the distinctive qualities of "an aesthetic object with human thought applied" are. This is a great form of exercise, and I think it will make people stronger? But given the "weakening" of the existing taste-landscape by blockbusters, we might be talking about lots of little refined "I don't touch that AI stuff" communities building aesthetic defenses with parochial snobbery (hey, it's worked for centuries in the past). That's probably better than most alternatives!
One little optimistic thing I noticed:
Midjourney users getting burnt out on currents of aesthetic similarity and no longer feeling the same dopamine highs as when they did their first "magical" generations, responded to by other users helping them rediscover the novelty in a) more powerful new versions (probably level off), b) mimicking more existing artists (er, educational?), c) using more features.
LLM generative content users are also wow'd at first, then hit limitations and have to spend a lot of time fine-tuning things in the way procedural generation artists have had to forever.
Great article Frank - this is my favorite of this series yet! I think there are kind of two thrusts here - one is your characterization of "taste" and how we all have it and how it will be our main weapon against any sort of super AI attention-weapon.
The only gripe I have is that I feel like you are overly rejecting of Chapman based, I guess *entirely* on the bad thought experiment. Sure, thought experiments matter, but they're not everything. And we know enough about the world to know that his overall point - the real concern is pools of power - is very likely to be the thing we ought to be concerned about. My feeling is that it's kind of just less sexy of a take. We gotta find a way to make talking about those boring old political battles, about democracy, or climate change, exciting. Maybe this is the real way in which "taste" is failing us.
It's the hallmark of lots of science fiction (and therefore, unfortunately, lots of thought from technocrats who grew up loving science fiction) that it extrapolates some current trend to absurd degrees, assuming humans will never notice or react to what's happening. A lot of it falls apart if you stop assuming we're all dumb frogs content to sit and never notice the water rising to a boil around us.
Kind of funny that people who want us to scale back and people who want us to forge ahead, technologically, can sometimes see humans in the same reductive way. Horseshoe Theory, but for Malthusianism!
I agree with you (and your son) so much that tragically I think I will have to write about it, because this category of error is one that I think many people I adore are prone to. I think there are even Deutschian explanations for *why* it's an error; I think often of his interview with Sam Harris in which he discussed the fact that most people quit addictive drugs because, in a word, they get bored.
Harris presses him on whether a "perfect drug" could be invented that would enslave people to hedonic satiation forever, and Deutsch is emphatic that it could not, for similar reasons to the ones y'all adduce above.
Anyway: fantastic piece, so exciting to read you on this stuff!!!
Nice, I'm glad you arrived at this point -- unsurprisingly it's what I started thinking about after part 2 and I was tweeting back when diffusion models blew up about a civilization-wide taste challenge. You know this is a subject I've dwelled on a lot over the years; I have long been interested in how the candy-coated mind parasites work, stick them in my own brain occasionally to test them, know how to fall back out, etc. I agree that you have the right call to battle, but I think you and James are being a little optimistic. Maybe that's just optimistic battle call rhetoric -- remember art, lads! Art is about humans... novelty is not JUST permutation! -- but I think we're still understanding HOW bad this is.
It's easy to look at attempts to harvest profit on mass taste -- the Beatles, church, Fortnite, everything Disney/Marvel puts out, etc -- and feel reassured that these things move in waves, people get bored of them, there's a counter-reaction, etc.
But your reaction here is exactly, precisely wrong in that "you are looking in the wrong direction, possibly at the past" kind of way:
"itβs really hard to fake SUPER POWERFULLY ENTERTAINING THING, like the show in infinite jest or the joke so funny it kills from the Monty Python bit"
I think we've seen by now that the significant difference with entertainment powered by machine learning is that it's hyper-tailored in the way blockbuster culture-bulldozers are not -- it fills in every other niche and crack in the same way that fan fiction is tailored by communities to fill in every possible crack not covered by a blockbuster (including "this isn't smart enough," I just saw HPMOR out on audiobook). But fan fiction is, by comparison, time-consuming to make, find (especially) and consume, and it works for people who really like to read.
So the taste challenge this time is more akin to an immune system having to resist bioweapons that tailor themselves to individual human cell receptors and can constantly mutate to find more statistical novelty in bounded possibility space. (The COVID pandemic furnishes commonly understandable metaphors and I'm surprised to find don't feel all that remote?) To put it another way, people get entranced by the dream-like quality of generative AI because it's a closer reflection of their own consciousness, individualized culture rather than mass, not a communication with an artist but with your own novelty/pleasure/mental-stimulation circuits.
If I'm going to compare my own foray into this particular taste vortex to things like falling deeply into a few different traditional game genres, into microtransaction-driving sim games, or trying to read several whole decades of comics, etc -- I find the consciousness-feedback-loop with this one much more engrossing, to a degree that's disturbing.
We already know people hypnotized by this effect to various extents, so the question is how long it lasts -- is "get high on a loop with your own subconscious" as dangerous as schlocky 90s sci-fi dreamed it would be, or will it wash over like another culture fad? It's likely to be variable, I guess? I see a lot of people stretching aesthetic muscles and trying to articulate for themselves and others what the distinctive qualities of "an aesthetic object with human thought applied" are. This is a great form of exercise, and I think it will make people stronger? But given the "weakening" of the existing taste-landscape by blockbusters, we might be talking about lots of little refined "I don't touch that AI stuff" communities building aesthetic defenses with parochial snobbery (hey, it's worked for centuries in the past). That's probably better than most alternatives!
One little optimistic thing I noticed:
Midjourney users getting burnt out on currents of aesthetic similarity and no longer feeling the same dopamine highs as when they did their first "magical" generations, responded to by other users helping them rediscover the novelty in a) more powerful new versions (probably level off), b) mimicking more existing artists (er, educational?), c) using more features.
LLM generative content users are also wow'd at first, then hit limitations and have to spend a lot of time fine-tuning things in the way procedural generation artists have had to forever.
Great article Frank - this is my favorite of this series yet! I think there are kind of two thrusts here - one is your characterization of "taste" and how we all have it and how it will be our main weapon against any sort of super AI attention-weapon.
The only gripe I have is that I feel like you are overly rejecting of Chapman based, I guess *entirely* on the bad thought experiment. Sure, thought experiments matter, but they're not everything. And we know enough about the world to know that his overall point - the real concern is pools of power - is very likely to be the thing we ought to be concerned about. My feeling is that it's kind of just less sexy of a take. We gotta find a way to make talking about those boring old political battles, about democracy, or climate change, exciting. Maybe this is the real way in which "taste" is failing us.
It's the hallmark of lots of science fiction (and therefore, unfortunately, lots of thought from technocrats who grew up loving science fiction) that it extrapolates some current trend to absurd degrees, assuming humans will never notice or react to what's happening. A lot of it falls apart if you stop assuming we're all dumb frogs content to sit and never notice the water rising to a boil around us.
Kind of funny that people who want us to scale back and people who want us to forge ahead, technologically, can sometimes see humans in the same reductive way. Horseshoe Theory, but for Malthusianism!