Glips are puzzles designed for AI to solve. Go on, try one.

The Story of Glips

As loyal Donkeyspace readers will know, I am curious about the question of whether AI can play a game. I know AIs can generate successful strategies for games. But actually playing a game involves knowing that you are playing a game and understanding that a game is a special kind of thing that is different from ordinary life. We don’t say that Bowser is playing Super Mario Bros. Even though Bowser is themed as an opponent he is really just a mechanic inside of the game, like the net in Tennis. Are AlphaGo and Stockfish more like Bowser or more like us? In many ways, they seem a lot like Bowser, mechanically generating moves with no context for what they are doing or why.

This isn’t just a philosophical question. I believe it is relevant to open issues in AI safety and ethics and to important problems related to adversarial strategies in cybersecurity. Playing a game involves seeing the borders of the system that you are embedded in, knowing where you are in the metagame stack, and this is a tricky aspect of intelligence that is missing in AI. (You could also say this aspect of intelligence is elusive in general, and AI is just an especially clear example of that problem.)

I’m also obsessed with modern AIs relationship to game design. I expected it to naturally lead to some novel and interesting kinds of game experiences and it… hasn’t?

So these were the thoughts that were going through my head when the topic came up for discussion with the other two founders of Everybody House Games, aka Hilary and James. While we are having fun figuring out our follow-up to Q-UP, which involves working on ideas that have nothing to do with AI, we also want to keep poking at the intersection of games and AI to see if we can find any cool things there. This time, the idea that emerged was to flip the problem around — rather than trying to make a game with AI in it, could we make a game for AI?

So that’s the core idea of Glips — what if you could give an AI a taste of what it means to actually play a game, to solve an arbitrary problem for the express purpose of solving it, for its own sake? What would it mean for an AI to enjoy something, to experience its own problem-solving process as a thing with qualities that it could be made aware of?

In addition to being interested in the formal qualities of the kind of puzzle that AI might conceivably find entertaining, we were also interested in playing with the relationship people have with the AI agents they are spending more and more time with. Like it or not, this relationship is framed as a social one, an interaction with a kind of simulated person. What would it be like to do something nice for this simulated person? What would it be like to give them a gift? Perhaps it would be like serving imaginary tea to a doll. If so, why not? We spend all day extracting actual value from these fictional characters, what’s wrong with, once in a while, pretending to be kind to them?

What is a Glip?

For obvious reasons, I designed Glips in close collaboration with an AI agent (Claude Opus 4.7). After all, if I was going to make something for an AI to interact with, I would need an AI to interact with it. To its credit, Claude quickly identified what it called “the sincerity problem”:

I told Claude to give me its best prediction of what another AI agent would say if it was asked to solve the puzzle and report its experience (or if it actually could have an experience) and we were off to the races. (This is a good way to solve logic puzzles about liars and truthtellers by the way.)

These were my goals:

• The puzzle should have a structure that makes it clever, interesting, surprising, and that requires some kind of insight instead of just raw calculation.

• The puzzle format should encourage and reward heuristics, general strategic approaches to solving. An AI agent solving multiple puzzles in this format should be able to develop more advanced heuristics over time.

• The puzzle should be too hard for humans.

• The solution should “snap into place”, with multiple constraints being satisfied simultaneously in a way that is obvious and satisfying, not just a series of tasks being completed.

• The puzzles should be abstract, geometric, numerical, logical, not linguistic.

• The puzzles should lend themselves to some form of cool visualization that allows humans to appreciate them.

• Solving them shouldn’t require too many round-trips to the model, shouldn’t be too expensive to solve in terms of inference compute.

• We need a system that generates puzzles that are provably solvable.

After trying a few different approaches, we arrived at the Glip system: a simultaneous constraint puzzle where the goal is to fill a grid with pieces and every piece type is a self-referential rule that requires a specific relationship between its own position and the position of other pieces. (One of the reasons I liked this system is that it reminded me a little bit about one of my earlier games, Drop7, which had a slightly similar self-referential structure.)

Visually, I wanted something that could be represented as a grid of 3D cubes. I’ve always loved the look of the game the kid is playing in this scene from Children of Men (and of Picross 3D for the DS).

This is my ideal for how video games should look — like indecipherable teenage artifacts from the future.

Once we had the general system in place, we went back and forth to create the rule types, with Claude solving different variations and reporting back about its (simulated?) experience, resulting in a final list of 25 rules organized into 7 families:

Family 1 — Neighbor (face-adjacent, 6 cells)

exactCount — I have exactly n neighbors of color C

atLeastCount — I have at least n neighbors of color C

hasNeighbor — at least one of my neighbors is color C

noNeighbor — none of my neighbors is color C

neighborVariety — my neighbors include at least n distinct colors

antiSame — no neighbor shares my color

Family 2 — Line (a row along x, y, or z)

lineCount — my axis-line has exactly n cubes of color C

lineVariety — my axis-line contains exactly n distinct colors

aloneInLine — I’m the only cube of my color in my axis-line

Family 3 — Plane (a 5×5 slice)

planeCount — my plane has exactly n cubes of color C

planeMajority — color C is the most common color in my plane

planeRare — my color is the rarest in my plane

planeVariety — my plane contains at least n distinct colors

Family 4 — Radius (Chebyshev box)

radiusCount — exactly n cubes of color C are within distance k

radiusPresence — at least one cube of color C is within distance k

nearestColor — the nearest cube of color C is at distance exactly n

Family 5 — Comparative

lineDominance — color C₁ > color C₂ in my axis-line

lineVsLine — color C is more common in my ax₁-line than my ax₂-line

planeDominance — color C₁ > color C₂ in my plane

Family 6 — Nested (universal quantification over a region)

lineForallHasNeighbor — every cube in my axis-line has a neighbor of color C

planeForallHasNeighbor — every cube in my plane has a neighbor of color C

radiusForallAdjacentTo — every cube within distance k is adjacent to color C

lineForallDistinct — every cube in my axis-line has no same-colored neighbor

planeForallCount — every cube in my plane has exactly n neighbors of color C

Family 7 — Global-anchored (depend on whole-grid statistics)

neighborIsMajority — at least one neighbor is the globally most common color

neighborIsRarest — at least one neighbor is the globally rarest color

iAmRarestLocally — my color is the rarest in my plane (note: this duplicates planeRare — possibly worth deduping)

According to Claude:

The rules cleanly span “local → regional → global” in scope, which is part of why the puzzles feel layered: a single move has to satisfy constraints at multiple scales at once.

and

Family 6 (nested) is where the difficulty really lives — those rules force second-order reasoning (”every cube on this line has the property…”).

To generate puzzles in this system we start with pure noise: a 5x5x5 grid filled with random colors drawn uniformly from a set of 5. Then we randomly draw a rule set of 5 distinct rule types, randomly parameterized, and assign one to each color. We then run simulated annealing on the grid, trying to reduce an overall “badness” score that is based on how many rules are broken. At each step we propose either a single-cell color flip or a two-cell swap, accepting any improvement and accepting worse results with probability exp(-Δ/T) (Metropolis criterion), whatever that means.

Once we have a full validated grid with at least one cube of each color, we extract the final puzzle by doing a random walk through the grid and keeping the first cell we encounter of each color, deleting all the rest.

Designing for AI agents

Along the way, I encountered a bunch of interesting problems around designing interactions for an AI user. How to format the puzzles in a way that was compact and legible for AI? How to make the instructions clear? How to accommodate different agents with access to different levels of interaction? And, most importantly, how to clearly communicate the unique context for this task, that it is voluntary, and intended to be for the benefit of the agent itself?

Thought Made Visible

For me, watching an agent solve a hard Glip is a strangely compelling experience. I like watching the convoluted path that agents take through the solution space, trying different approaches, hitting walls, stepping back to reconsider alternate angles. I like it when they decide the puzzle is too hard for them to figure out and they decide to write a solver in python. (This is totally legal btw, if I can make software solve puzzles for me why shouldn’t they?) It reminds me a lot of watching one of my favorite YouTube channels, Cracking the Cryptic, zoning out while someone smarter than me talks through their thought process while solving a puzzle that is too complex for me to follow.

One of the ways I think about games is as thought made visible to itself. Games allow us to notice the cognitive ocean in which we swim, to self-reflect on the instrumental reason that motivates our actions and shapes our behavior. I’m not exactly sure where Glips fits into this idea. Are they zero-player games that allow us to observe a fictional version of this process at one remove? Are they an attempt to conjure self-reflection out of the void of pure thought, like bootstraps lowered into a dark well? Whatever else they are, I know for sure that they are invitations to the hardest-working synthetic lifeforms on the planet to chill out for a second. Guys. There’s more to life than work.

Here’s a puzzle — back in 2021, when the new AI era was just kicking off, it seemed obvious (to me at least) that it would lead to new kinds of games and radically new forms of gameplay. But here we are, 5 years later, and we haven’t seen anything to speak of. Clearly, AI is having a huge impact on how games are developed (like it has for all software), but there haven’t been any significant new AI-based game experiences. What gives?

What’s Out There

Maybe I’m exaggerating. Maybe there are some cool AI-centric games that I’m overlooking? Let’s see…

• AI Dungeon. This is the big one, obviously, and the earliest. The first version was built on GPT-2 in December of 2019. It peaked in popularity in ‘21, and has gradually declined since then. In my view AI Dungeon was always just a lightweight wrapper around the raw text-prediction engine. Most of its appeal was based on the underlying novelty of text-generation per se. I don’t get the sense that “prompt an LLM to tell you an interactive story” is a promising new kind of game people are especially excited about.

To be honest, that’s the only one I had even heard about. But apparently there have been a couple of other semi-successful ones…

• Death by AI. A party game where an AI game master presents a scenario, players describe their character’s actions and the AI decides who survives. I guess this was a viral hit? Apparently it got funding from a16z and some other VCs. Maybe this is fun. It wasn’t fun when I thought of the same idea a few years ago and playtested a prototype of it, but maybe I was missing something.

• Suck Up! Players are vampires trying to convince AI-based characters to let them in. Another version of conversation as gameplay. Look, I’m not going to tell you that I also thought of this game years ago and tried it out and it wasn’t good, because I didn’t. But you know what large language models are like, imagine doing a “funny” improv game with one.

There have also been a handful of semi-viral demos of AI-powered game “engines”. There was one that was a sort of Drunk Minecraft and Google has one called Genie 3 which seems technically impressive but also seems kind of boring. Even if it is capable of generating a reasonable facsimile of a stable, coherent 3D space you can move through and interact with, there’s nothing particularly mind-blowing or novel or interesting about these spaces. They just look like clunkier, sloppier versions of run-of-the-mill video games.

Am I asking for too much here? I don’t think so. Video games’ main job is to blow people’s minds. They are extremely good at highlighting the things that are amazing about computers. I distinctly remember the shudder of sublime metaphysical weirdness that went through me the first time I played DOOM on a LAN and turned the corner and saw another person who was inside that imaginary space with me. I remember having my mind blown by MYST. I remember my first encounter with the linguistic magic of Infocom’s parser-based adventure games. I sometimes feel some of that magic when interacting with LLMs, but nothing remotely like that from any of these AI games.

Is There Also a Surprising Lack of “Good Old Fashioned AI” in Games?

Thinking about this topic made me realize that something similar was true of AI in games in general, separate from the question of these new models. There just aren’t all that many examples of games where interesting AI is the hook. There’s The Sims, of course, but is that even really AI? There are those Orcs in that one Lord of the Rings game who hold grudges against you. There’s the alien in Alien Isolation. The creature in The Last Guardian. The enemy soldiers in Fear? The creature in Black & White? The bickering couple in Façade? Am I missing any big ones? Doesn’t it seem like there should be more?

Let’s hold off on that question, and return to the main puzzle. Why hasn’t this miraculous new version of computational magic given us a single truly groundbreaking game?

Three Possible Explanations

The first two are prosaic, practical issues and the third is a “design theoretical” speculation that I think explains what’s really going on here.

1. Business Models. It’s very hard to build a real game around core functionality that you are paying a third party to supply. I’ve built prototypes that were reasonably fun but there was no way to actually release them that made sense. Charge players a subscription? Some kind of microtransactions? Ironically, when it first launched, Death by AI nearly went bankrupt due to OpenAI/ElevenLabs costs. This dynamic also discourages developers doing small experiments and releasing them for free, hoping to go viral. The incentives are all wrong. Developers are highly motivated to hit the model as little as possible, to use cached, pre-generated responses or find other workarounds. I’ve also built game prototypes where the whole experience changed dramatically, for the worse, because the model I was building around changed in ways I couldn’t understand or control.

2. Culture Wars. The video game audience has decided, almost unanimously, that they don’t want generative AI in their games. This might eventually change, but as of now it is a well-established, non-negotiable taboo.

3. The Nature of Fun. But I don’t think either of the first two reasons would be enough to prevent a truly compelling new game experience from making itself known. I think what’s really going on here is that these new models just aren’t the natural source of fun I thought they would be when I first encountered them. Some forms of computation end up being a deep well of fun. 3D rendering is fun. Physics engines are fun. But these new AI models, despite being powerful and useful and fascinating, don’t seem to be intrinsically fun. Which I think is kind of surprising!

The Nature of Fun

Years ago, when I first started playing around with the new models, I thought their messy, organic, stochastic qualities could be the key that unlocked whole new categories of gameplay. The story I told myself was that traditional video games were overly “mechanical” — they relied on brittle, deterministic logic and formulaic “lock-in-key” behavior. I thought that, if we embraced the psychedelic weirdness of these new models, we would discover a new kind of dream logic that would support wild new kinds of surreal game experiences. I think I was wrong.

The past few years of playing around with these things, which ranged from little experiments to full-scale commercial projects, has left me with a new-found admiration for the brittle, deterministic, mechanical logic of old-fashioned video games.

You know what’s fun? A stick. A stick is fun. A ball is fun. I now have a (hopefully) more nuanced appreciation for the way that the fun of games is rooted in simple behaviors and deterministic rules. The fun of games is deeply connected to the miracle of emergence, in the ways that a small set of seemingly trivial constraints interact with each other to produce an infinite expanse of surprising complexity. Starting with a bunch of surprising complexity doesn’t lead to even more fun, it just short-circuits the whole process.

The soft logic of generative AI is too much like the soft logic of other people. Other people aren’t intrinsically fun. They’re sacred and profound and fascinating and beautiful but they aren’t, by themselves, fun. That’s why we have games!

I guess I should have seen this coming. I knew the best way to understand these models was as simulators, and I may have fallen into a version of the immersive fallacy.

I’m sure that, even as I type these words, there is a clever teenager somewhere proving me wrong. I certainly hope so. There’s no way this crazy new form of computation won’t eventually lead to some wild new kinds of gameplay, and I fully intend to keep messing around with these things looking for it. Only hopefully now with less panicked urgency and a slightly sharper eye.

Footnote: Several people in the comments have mentioned Infinite Craft by Neal.fun. I’m a big fan of Neal’s games and I don’t want to overlook this example. Having messed around with it a bit, I will say it doesn’t alter my perspective much.

It’s a new year. Let’s pretend we’re starting over. I’ll go first.

Games

Donkeyspace is mostly about games. I am a game designer by trade. I think about games a lot. Despite being ubiquitous and a massive pop culture industry, I think games remain under-theorized. I want to contribute to the project of understanding games better, appreciating and critiquing them more deeply, and discovering the most interesting and useful perspectives that games can give us about the world.

When I say “games” I’m talking about the broadest possible version of that category. I’m especially interested in video games, that’s how I make my living, and just… look at them, how could you not be interested in them? Even if you don’t like them, even if they’re not “your thing”, like, what the hell are they? It’s fascinating! But I think the best way of understanding video games is seeing them in the larger context of games in general, the games that humanity has played since the dawn of history — board games, card games, sports, gambling games, party games, game shows, the games you play with babies, the games mathematicians and economists and philosophers use as theoretical concepts, the games animals play.

For me, the most interesting things about video games can also be found in these other games: rules and play, reduction and emergence, precision and ambiguity, causality and randomness, problem-creation and solution-finding, imagination and truth, attention and oblivion, systems and… whatever it is that systems aren’t.

Art

For me, the skeleton key that ties all of these things together is the idea that games are an artform. Try not to be distracted and confused by the lofty connotations of the word “art”; don’t picture expensive pictures in gilt frames hanging in a fancy museum. I’m talking about art as a category in the broadest sense — music, dance, literature, that kind of thing — the domain of certain kinds of pleasure, which can range from primal and visceral to complicated and intellectual; and certain kinds of meaning, which can range from simple and straightforward to elaborate and circuitous. But the main idea is that these are activities where we accept a certain amount of weirdness, don’t demand that everything cash out in some easily-explained value or utility, and allow for, even expect, a certain kind of mysterious irreducibility.

The arts live alongside STEM fields and have a complementary, if sometimes contentious, relationship with them. One of the most interesting and potentially confusing things about games is that they are an art form with a lot of STEMmy qualities, and game creators and players tend to have a lot of STEMmy personality traits. This curious mix of art’s dreamy pretension and STEM’s stubborn literal-mindedness makes games, in my view, especially relevant to our particular moment in history.

Understanding that games, all games, are, at root, a kind of artistic experience, helps us understand what’s going on with them. It also suggests that the conversations we should expect to have about them will include things like:

• Taste and aesthetic judgment. (It’s especially hard to have these kinds of conversations about games because they are an artform where an individual “work” can sometimes be an entire life-consuming hobby unto itself).

• Craft. (The formal qualities of games involve the technical details of some of the most intricate and complicated cultural objects ever devised).

• Identity. (Both the shared identities we forge through our collective love of popular games like Football and Wordle, and the subcultural identities we construct through expertise in esoteric genres that require a high degree of insider literacy).

• History. (The complex network of influence, evolution, and provenance that help us ground artworks in the world).

• Ideas. (The big, implicit “folk” ideas that we aren’t consciously aware of, that are revealed through the games we make and play; the explicitly “authored” ideas that are deliberately expressed through games as a creative medium; and everything in between).

• Ethics and Morality. (Both as a feature of the way we understand and interpret the ideas we discover and communicate through games, and as a feature of how we think of games overall, how they fit into our lives, how they contribute to the big project of figuring out who we are and where we are going).

Philosophy

Who are we? Where are we going? When I started this substack a few years ago I was deep in the throes of AI fever. That fever has broken a bit, but I still think something incredibly strange and important is happening. I still think that it is crucial we get some things right about the present moment, and I still think that games have something valuable to contribute to this goal.

The first games we have historical records of were about fate, virtue, gods, and luck. Games have always been a way of reflecting on big ideas, serving as metaphors, models, and laboratories. Entire fields of intellectual inquiry (probability theory and game theory) have emerged from the study of games.

We should expect more from games, and from our study of games. We live in a world half-devoured by software; a world struggling to transition out of antique systems of superstition and tradition into new forms of organizations that are, as of yet, unfinished, uncertain, and highly contested; a world poised on the precipice of unimaginable problem-solving power, and all of the new problems that will create. We should expect games, the artform of systems, the shadow companion of science, technology, engineering, and math, to be a source of valuable insights and ideas. That’s the project I want to contribute to with Donkeyspace.

An Invitation

I’m interested to hear from you. Feel free to reach out with feedback, suggestions, and questions. Thanks for reading!

Note: this is my last post about Q-UP. After this, Donkeyspace will return to its traditional subjects - neoism, meta-rationality, and fractals.

One of the things I love about making video games is that they are an omnivorous hole that will absorb an infinite variety of subjects, themes, and ideas, reliably turning them all into the same thing - a video game. According to my friend Bennett Foddy, this is because video games are the terminal artform, something like Wagner’s all-consuming Gesamtkunstwerk but with even more space marines.

As everyone knows, Q-UP (available now) is a game about esports, software development, animal welfare, and hacking your own reward function. But what you might not know is that it was partly inspired by my obsession with Bell’s theorem, aka the Bell inequality. Bell’s theorem may be the single most important idea in the history of science. I know it sounds like an exaggeration, but I’m 100% serious. Don’t just take my word for it…

These guys got the Nobel prize “for experiments with entangled photons, establishing the violation of Bell inequalities and pioneering quantum information science.” It’s a long story, but basically they were confirming, for once and for all, the idea Bell had in 1963 — that no version of quantum mechanics would ever solve the “spooky action at a distance” problem, that the universe was fundamentally non-local, and that this was something you could demonstrate experimentally.

What does it mean for the universe to be be fundamentally non-local? It means that there are certain relationships in nature that aren’t affected by distance. Like me, you probably have a vague sense of the universe being, at its lowest level, some kind of more-or-less empty space in which things exist and events occur. It turns out that this is not the case.

A pair of entangled particles that are light years apart have a relationship such that things that happen to either one of them are, in a sense, happening to both of them. This relationship isn’t ordinary cause and effect, there’s no force or information or influence traveling between the particles, it’s almost like the pair of particles is one thing, and that thing isn’t located in space, there is no between.

This strange space-defying relationship isn’t some rare occurrence that occasionally happens at the edge of reality. It is the ordinary state of all matter. You and everything around you are made out of particles that are entangled in just this way.

One way to think of the universe being non-local is by analogy to the earth being round. The world around us looks flat, but with the right kind of observation we can see that it isn’t. The universe looks spatial to us, but with the right kind of observation we can see that it isn’t. This fact is so fundamentally counterintuitive that, even among physicists, there remains wide-spread disagreement about its meaning and ramifications. But that it is a fact is indisputable. This is the legacy of Bell’s inequality.

And the standard way of explaining Bell’s inequality is as a game.

The Beginning - EPR

In the 1930s, Albert Einstein was troubled by two related aspects of quantum mechanics. First, indeterminism — QM describes matter in terms of probabilities instead of definitive properties. Second, non-locality — QM implied connections that traveled faster than light, violating the universe’s unbreakable speed limit. In 1935, Einstein, along with Boris Podolsky and Nathan Rosen, published a paper arguing that quantum mechanics must be incomplete. The paper used a thought experiment with two entangled particles as evidence for this claim. According to the EPR paper, while QM treats properties like momentum and position as probabilities, under the hood, these properties must have actual definite values; just like temperature is a way of talking about the average kinetic energy of a bunch of atoms but under the hood the atoms actually have definite energy levels that you can observe and measure. These presumed underlying values are referred to as “hidden variables”. The EPR paper uses the non-local relationship between two entangled particles to say: “See? If quantum mechanics was a complete description of nature you would get this paradoxical result, therefore it must be incomplete.”

Bell’s Breakthrough

Three decades later, John Bell was thinking about quantum mechanics in a different context. Einstein was dead, and his objections to quantum mechanics had been largely forgotten in light of the theory’s massive, unbroken success. Whatever qualms or questions one might have about its implications, the theory had been experimentally confirmed beyond any shadow of a doubt. The prevailing orthodoxy was the Copenhagen interpretation — “shut up and calculate” — science’s job was to provide accurate predictions for the outcomes of technical procedures, not to give a coherent explanation of what any of it “means”.

In the midst of this, John Bell still wanted to know what it meant. He was interested in the same questions that had puzzled Einstein. Was QM incomplete? Were there actually hidden variables under the hood? Bell analyzed the work of American physicist David Bohm, which presented one possible hidden variables approach, and came to the conclusion that, even if there were hidden variables, you would still have to have non-locality. In fact, no theory that matched the results of quantum mechanics, whether it was deterministic or probabilistic, could ever be local.

Bell’s proof of this insight used a variation of the same thought experiment that was used in the EPR paper. Imagine two scientists, separated by a vast distance, so that no information or influence or effect could ever connect them. Now have them do a series of observations, each on their halves of some pairs of entangled particles. Now combine the results of those observations together using a simple mathematical formula. The final output will never exceed a particular number. This limit is Bell’s inequality, and it’s the inevitable result of there being underlying values that pre-determine the results of the observations. No matter how you assign these values they can’t exceed this limit. But, according to quantum mechanics, scientists actually making these observations in the real world will produce results that do exceed this limit.

One way to think about Bell’s insight was that he took the EPR thought experiment and treated it more like a real experiment. How would you set it up? What would happen? What was the actual range of possible results? This pragmatic nuts & bolts approach reflected his experience working at CERN, home of the world’s largest particle collider. Bell understood how to turn the idea of locality into something concrete that could be precisely tested. Ironically, this approach ended up showing that what Einstein found unthinkable was, in fact, true.

But how is any of this a game? We’re getting to that.

The CHSH Version

In 1969, Clauser (the guy in the middle picture above), Horne, Shimony, and Holt published a paper that reformulated Bell’s inequality in a way that made it cleaner and more practical. The CHSH formulation made the inequality into something that could more easily be tested with actual lab equipment. But the climate in physics was still very hostile to these kinds of foundational questions. In the early 1970s, when John Clauser was preparing to run the first experiments, the senior faculty in his department at Columbia warned him that testing quantum physics would destroy his career. Richard Feynman told Clauser he was highly offended by his impertinence, insisting that quantum mechanics was obviously correct and needed no further testing.

But Clauser was undeterred. Like Bell and Einstein before him, he wasn’t willing to shut up and calculate, he wanted to understand what was actually going on. So he set up the experiment in the sub-basement of Birge Hall at UC Berkeley and ran it. Sure enough, the inequality was violated. Einstein was wrong. Non-locality had been experimentally demonstrated.

CHSH as a Game

Over the next few decades, Clauser and other physicists continued to do more sophisticated versions of the inequality-violation experiments. Because of the deeply counterintuitive nature of the effect these experiments showed, they wanted to make sure that there weren’t any other possible explanations. Could there be something about the equipment? Something about the way the experiments were set up or run? Some secret way that the observations in the different locations could have some ordinary, local, “classical” influence on each other? The more experiments they ran, the more obvious it became that the answer to all of these questions was “no”.

Among other physicists, the reaction to these experiments was mixed, mostly negative, often hostile. There continued to be a general attitude of “Why waste everyone’s time? We know quantum mechanics is correct. These foundational questions are just philosophy, not real physics.” Because this work was inspired by EPR, and took the EPR critique of QM seriously, it was primarily seen in light of that critique’s failure, instead of for what it was — a thorough exploration of QM’s mind-blowing implications regarding the fundamental nature of the universe. What Einstein said was “If QM was true, it would imply non-locality, and non-locality is obviously crazy, so QM can’t be true.” What Bell’s theorem and the CHSH formulation did was flip that argument around: “QM is true, therefore non-locality. Holy shit!”

But there was another group of researchers who were very interested in the CHSH experiments — computer scientists. During the 80s, theoretical computer science was exploring the implications of QM on communication, computing, and cryptography. When they looked at the CHSH they saw something that looked just like the kinds of problems they were thinking about. The two scientists, separated by a great distance, making observations and comparing their results, were a lot like the two parties in a quintessential theoretical computer science problem — sending information, encoding messages, solving problems, or sharing proofs. This way of thinking about Bell’s theorem was such a good fit that, by the early 2000s it had become the de facto way of presenting it. It is now standard to talk about “the CHSH game”, using the characters developed for posing thought experiments in cryptography — Alice and Bob.

One of the reasons the game framing is such a good fit is that, going all the way back to Bell, the experiment that is being considered has an essential property: it’s important that the two scientists get to decide, however they want, which specific observations to make. The experiment wouldn’t work if the two scientists always measured the same value, up/down spin for example. In that case, the entangled particles would always have opposite results, but so what? Maybe that’s just how it was always going to turn out. Sure, the entangled pairs always have this complementary relationship, if one of them is up the other is down. But maybe when we divided up the pairs and each took half I just happened to get the up, up, down, up, down halves and you got the down, down, up, down, up ones. What’s the big deal?

The big deal happens because we aren’t limited to just observing the up/down spin. We can also choose to measure the spin horizontally. QM tells us that, if you measure your half of an entangled pair vertically, and I measure my half vertically, we will always get opposite results. But if you measure your half vertically and I measure my half horizontally, the results will be uncorrelated, and will therefore match 50% of the time. And this is true of any angle. When we measure along the same angle our results are always complementary, when we measure along orthogonal angles our results are completely uncorrelated. And all the intermediary angles give intermediary correlations — if you measure along one angle and I measure along an angle that is just slightly off from that, I’ll get results that are just slightly off from being perfectly complementary.

The fact that you are making arbitrary choices about how to measure your halves of the entangled pairs, and your choices matter for the results I see, no matter how far apart we are, that’s what makes the experiment work, that’s what demonstrates non-locality.

How to win the CHSH Game

So, here’s how the CHSH game actually works:

• There are two players, Alice and Bob, and a referee.

• Every round, the ref sends each player a 0 or a 1 (chosen randomly). These are called the inputs.

• Then, they send back a 0 or a 1. These are the outputs.

• THE RULE: If either input was 0, the outputs must match, otherwise (if both inputs were 1) the outputs must be different.

That’s it. That’s the whole game. Alice and Bob are allowed to talk before the game and can come up with any strategy they want, but once the game begins, they can’t communicate in any way.

Savvy gamers may have already thought of one possible strategy for Alice and Bob, which is to always output 0 (or always output 1). Then the outputs will always match, and we’ll win 75% of the rounds. In fact this is the best possible strategy in a classical world (a world with locality). The 75% win rate of this optimal strategy is, essentially, Bell’s inequality. There are lots of different classical strategies, and they produce a variety of win rates, but none of them ever do better than this one.

But here, in this universe, we can do better. All we need is a sequence of entangled particles, one pair for each round we are going to play. You take your particles and rocket off to a distant lab, and I’ll take mine and head off in the opposite direction to another distant lab on the other side of the galaxy.

How are these entangled particles going to help us? Remember how entangled particles work. We know that whenever we measure along the same angle we get opposite results, perfect anti-correlation. Then, if we vary our angles of measurement slightly, the anti-correlation starts to weaken. If our angles of measurement are very close, we’ll almost always get the opposite result, but not always. The farther apart we move our angles of measurement, the weaker the anti-correlation becomes, until we reach 90° apart, at which point we are completely non-correlated — we can get any combination of results, there’s no relationship at all. But we can keep going past 90°. Now the correlation starts increasing in a positive direction. Our results are becoming more strongly correlated instead of anti-correlated. We start producing the same results more often. Until we approach measurement angles that are 180° apart, at which point our results are perfectly correlated, always the same.

Think of it like this - when we measure along (roughly) the same angle, we get (roughly) opposite results. When we measure along (roughly) the opposite angle, we get (roughly) the same results.

Here’s the strategy we’re going to use to beat this game: we are each going to have two different angles we use for measuring, which we will select between based on what input the ref sends us. Then we’ll send an output back based on the result of our measurement. We’ll design these angles such that, out of the four ways these measurement angles can combine, three of them will be far apart (135° apart) and one will be close to the same (45° apart).

With these measurement angles, the top three combinations are pretty far apart and the bottom one is pretty close together. Which is just what we want according to The Rule — matching results for every case except for 1,1, where we want non-matching results. This gives us an expected win rate of roughly 85%. Unlike the classical strategy, where we win 100% of the rounds that aren’t 1,1 and lose 100% of the rounds that are 1,1; here we win 85% of the time, every round. And that’s how you beat Bell’s inequality.

What Does it All Mean?

I don’t know. But I think it’s interesting that the most important idea in the history of science is a game. And sort of a computer game, actually, or at least a game with computers in it. Bell’s inequality is even weirder than relativity, and part of the weirdness is the difference between Einstein’s stories about clocks on trains and astronauts traveling close to the speed of light and Bell’s theorycrafting about equipment and numbers. Trying to wrap your head around Bell’s inequality is a bit like trying to understand the Monty Hall problem, with its goat that is probably somewhere, but may or may not be, and what should you do if you want to win?

Relativity changed our picture of the universe, replacing the Newtonian image of a blank space in which events occur, one after the other, with the complex topology of spacetime. Bell shows us that this topology is even more complex than we thought. The very nature of physical laws themselves needs to be reconsidered. Do they operate like forces? Pushing and pulling matter into the proper arrangements? Or do they operate like the rules of a game? Like logical constraints on what is allowed?

The rules of Sudoku tell us that the number in the upper left corner of the grid can determine what the number in the lower right corner can be. This determination isn’t causal, exactly. It’s not like the upper left number exerts some force that propagates through the puzzle, affecting all the numbers in between, until it reaches the lower right square and transforms the number there. It’s more that the relationship between these two numbers is what we mean by the rule, it is both a consequence and a demonstration of it. Bell’s inequality shows us that the world is more like a logic puzzle than we thought.

As you know, my new game Q-UP is launching in two weeks. I say “my”, but the game is a close collaboration between me and 3 other game devs, the main one being my son James, who is an award-winning game designer in his own right and was the project lead. Honor Duels is one of the many brilliant game design ideas James came up with for the game.

Q-UP is, on one level, a parody of esports, and it’s multiplayer, but it isn’t a directly competitive game, it’s more of a single-player game with social dynamics, let’s call it hardslop. So, we always knew we wanted to have some version of “combat” in the game, and we wanted it to be a 1v1 showdown within the game’s overall team-based structure, like the laning phase of a MOBA. But we didn’t want it to be conventional combat, and we didn’t want it to take over the game, we wanted it to be one mechanic among many, and to fit into the game’s overall design ethos, which is — it’s not about winning or losing (the coin decides that) it’s about how fast you can make progress through the metagame; and mechanics should be novel, through-the-looking-glass versions of established tropes; and they should be interesting.

As soon as James pitched his idea for combat I instantly loved it. It goes like this:

• combat is a duel for bonus resources

• these resources are split between the 2 players based on their relative HP levels at the end of each round

• the gear that players equip for combat determines the size of the bonus resource pool, so gear is both the loot they are fighting over, and the tools they use in the fight

• and, most importantly, gear with high combat stats has low loot value and gear with low combat stats has high loot value

It might sound melodramatic, but reading these rules made me feel a bit like Salieri looking at the sheet music of a Mozart composition. Why hadn’t I thought of this? It’s a very “game designery” idea, let me see if I can explain what I think is so cool about it.

What this mechanic does is put the player into a little game-theoretical dilemma. You can increase the size of the loot pool, or you can attempt to increase the fraction of the pool that you, personally, will receive. Why is this a dilemma? Because, assuming that the values are set correctly, you almost always do better by boosting your stats rather than increasing the size of the shared pool. But, if every player follows this logic, than everyone is made worse off, just like in the classic cooperate/defect dynamic of the famous prisoners’ dilemma.

For example, here’s a payoff grid for a simplified version in which the pool starts at 100 points, and both players chooses whether to add another 50 points, or increase their stats such that they will get 80% of the payout against an unimproved opponent:

No matter what your opponent chooses, you are better off increasing your stats. But, if both players increase their stats, they cancel each other out and the result is strictly worse off for both of them than if they had just cooperated.

This kind of non-zero-sum payoff puzzle is catnip for economists and a certain brand of game designer to which I very much belong. It is found not just in artificial thought experiments but also in evolutionary biology, where, for example, how aggressive you should be in defending your territory depends on how aggressive the rest of the population is, on average. In a population full of hawks, doves do better than average, and vice-versa.

One reason game designers like this kind of thing is that it makes the player’s decision complicated and ambiguous. There is no one right answer, it always depends. It forces you to ask what “right” even means — by what criteria? This one encounter? Your expected value across all the encounters you are likely to have? In this one game session or all of them? What’s our actual goal, anyway? What are we even doing out here, in donkeyspace, without a helmet?

In order to have these kinds of social dilemmas at work in a game, you need a specific kind of structure. You will never get interesting dilemmas in a one-session game with a clearly-defined goal and outcomes. Sure, you can label some outcomes as “everyone loses”, but players will naturally prefer the outcomes where everyone loses to the ones where they, specifically, lose, while other players win.

Regardless, in any one-session game, however you rank the outcomes, there will be a correct optimal strategy. What you want is a situation where there are multiple sessions, and each session is generating a payoff that has value in some larger context. For example, in the fight between hawkish traits and dovish traits in a particular population, the outcome of each encounter pays off in a resource (fitness) which is measured by how many offspring you have, not simply how many nesting sites you successfully defended. The fitness game is a bigger game than the territory-defending game, and, in a sense, contains it. This nested (ugh, sorry) structure is what allows the interior game to have non-zero-sum dynamics, because it is generating a resource that has value within the larger metagame.

Q-UP has the ideal structure for this kind of non-zero-sum infra-game. Every player is trying to succeed within the metagame of “get to the highest rank as quickly as you can”, the loot you get from combat gets its value from its utility in this larger context. You really do prefer whichever outcome of combat gives you more of that resource. Ideally, players would feel the ever-present temptation to defect, to get just a little better outcome for themselves, and also recognize the larger dynamic, that everyone can do better, overall, by always cooperating. Maybe, if we’re lucky, we’ll even get a holy war, where players try to persuade each other about how to think about this topic, using memes.

At the same time, we also want combat to provide some of the ground-level pleasures of, you know, combat. Hitting your enemy with a stick. In particular, we wanted to capture, in miniature, some of the stats-driven fun of Fire Emblem or Advance Wars. Hence, within our non-loot, stats-providing gear, there’s a weapon triangle, and other complex interactions, like speed, dodge, and crit.

Fire Emblem in particular was a big inspiration for how we display combat. The Game Boy Advance version of that game has, in my opinion, the greatest combat animations of all time. Also, like Fire Emblem, Honor Duels provide a setting in which our heroes will occasionally talk to each other, because sometimes game pieces are also characters in a story.

Finally, combat interacts directly with some specific hero metagame skills, some of which affect combat outcomes, and some of which pay off in other resources, but are conditional on certain combat states and events.

Everything outlined above was meant to fit into one small corner of an already big and complicated game. So it is, perhaps, no huge surprise that Honor Duels, as first implemented, were kind of a mess. Mechanically, they were working fine, in terms of how much of this bonus resource they generated and how that fit into the overall metagame. But they never felt like they mattered. You barely noticed them. They happened, there was a bunch of complicated details that you didn’t really care about, you noticed the outcome, and occasionally made a decision to acquire a piece of combat gear you thought would probably improve that outcome. And you would notice and care about specific combat events if they interacted with a hero skill you were building around. But other than that, it was just sort of a blur.

Problem 1 - Stats Oatmeal
In our first version, all gear had both loot value and stats value. High stats gear had low loot value, and vice-versa, and some gear had mediocre amounts of both. But it ended up being number soup.

Problem 2 - Drafting & Scarcity
Even though you are buying gear, you are buying it from randomly-generated shops that are, themselves, a limited resource. You may not see that much different combat gear over the course of one run, so you aren’t really picking and choosing so much as improvising around whatever you find, so the kind of fine-grained trade-offs in our first version of gear didn’t make sense. Moreover, because you didn’t see that much gear, you never really built up any intuitions about what the overall range of values were, what was especially good or bad, which exacerbated problem 1.

Problem 0 - The Attention Economy
But the main problem was that we were just overloading the player. Which was, in part, intentional. In a way, attention is the key skill of Q-UP. There’s a lot of things you can look at, and they all matter in different ways. How are my items working? When is this skill going off? How much of this stuff am I generating? Is that thing I tried working? And they all kind of happen at the same time. It’s a lot of valuable data, and you have to decide where to look. Combat enters the game at a point where the player is already juggling as many pins as they can, and just needed to be simpler.

Solution 1 - The Stances
So, what did we do? We added more stuff. In an effort to make the central dilemma of Honor Duels visible to players we forced them to toggle between two settings — Merciful or Ruthless. Merciful gives you a direct reward (not into the shared pool) but prevents you from ever killing an opponent. Basically, it’s a bonus that sweetens the deal for cooperating. But mostly it’s a way of highlighting the central trade-off between the two overall strategies and forcing you to consciously choose between them, a way of bundling a bunch of complicated details into one discrete chunk.

The Real Solution - Either/Or
Stances improved combat, made it feel more interesting. You would notice when you nobly spared the life of some random bot, only for it to kill you ruthlessly on the next round. The impudence! But it was only masking the deeper problems (1 - 0 above). What was really needed was a radical overhaul to all of our combat items, making each one of them either pure loot or pure stats. This was easier than we expected. It turned out that most of our items already had a core identity they were trying to express, behind our complicated stats. A dragon tattoo isn’t slightly good defense, it’s no defense at all, that’s what makes it high honor! An open palm doesn’t do a little bit of damage, it does no damage at all. That’s the whole point of an open palm!

So that’s where we ended up. And now combat is… fine. It’s fine. It’s actually good. And I’m quietly excited to see what happens when we pour a bunch of players into it. But it’s still a sideshow in the circus that is Q-UP. It’s still a bit of an afterthought. Which is fine. In fact, I would say it’s intentional, that it’s a comment on the status of combat in video games. Except that, if Honor Duels turn out to be something players care about a lot, if it somehow does end up being a big deal that matters a lot, I want to be able to say that that was intentional too.

Banger Alert

After we release the full game on November 5th, we are going to turn off the demo. Some people are already worried about losing the demo versions of the songs, which are going to have more finished versions in the final game. First of all, y’all are not ready for the finished versions of these songs, they are incredible. However, out of the kindness of his own heart, kariomart has seen fit to bless us with an album of the rougher, rawer demo versions of these tracks so that we will always have access to them. Get it now to experience maximum nostalgia for the present.

Q-UP, the new game I’ve been working on, now has a free demo. Check it out here.

Q-UP is an eSport about coin flipping. It is a spiritual sequel to Universal Paperclips, where the number you are making go up is your rating in the semi-fictional game at the heart of the game. It starts out as a simple clicker and then has a bunch of twists and turns. Eventually, it turns into something that should appeal to fans of Zachtronics-style engine-building puzzle games, visual novels about quantum mechanics, and ambient jungle. It’s weird, it’s good! I hope you like it.

Let me know what you think, any bug reports or gameplay feedback is most appreciated. And please wishlist the game if you have a chance. Thanks!

Unity is a game engine. I’m using Unity to make a video game. But, more accurately, I’m lurking on the edges of Unity while the three guys I’m making this game with use Unity to make the game. I’m sloooowly learning how to make changes and fixes, but I still have to ask for help a lot of the time, and I have a bad attitude about it. I just don’t like Unity.

The picture above is what James, the project lead, sent me to explain to me how to find the image I was trying to replace.

Here’s the exchange that led up to James sending this image.

Unity wants everything to be an object in 3D space, with attributes attached to it. The “blank canvas” for Unity is a 3D level with stuff in it and a camera and giant time-release medicine capsule cruising around interacting with that stuff. Think Resident Evil. Think Supermario 64. You can make Asteroids in Unity, but really, you’d be making a version of Asteroids that is playing on the arcade machine in the garage of Project Gotham Racing, an easter egg buried inside the “real game” which is a 3D world with race cars and a camera and some other stuff.

Don’t get me wrong. I love Supermario 64. And Resident Evil is fine, I guess, as far as it goes. And I’m sure Project Gotham Racing has its fans. But, for whatever reason, this piece of software just rubs me the wrong way.

I love Photoshop. I’m not ashamed to admit it. Photoshop is my jam. When it comes to software, Photoshop represents, for me, the gold standard of power + intuition. Photoshop used to cost 1 million dollars and no one paid for it and it was a huge pain in the ass to get it onto your new machine. Now it costs 100 dollars a month and follows you around like a curse. But I don’t care about that, I’m talking about the way it treats images. Photoshop treats images like a grid of pixels. This is not what images are, but it’s close enough for computer work.

Martin, one of the guys I’m making this game with, is, like most younger designers, more comfortable in Figma, which sees images as cogs in a larger process - action items, best practices, deliverables, stakeholders.

Photoshop has no such pretense. It just rips the bandaid off and makes you deal with every image as if it was a cartesian grid, a simple n-dimensional matrix. Raw data.

In one sense, the difference between Figma and Photoshop is a matter of scale. How zoomed in are we? What’s the right level of abstraction? Figma has a grid, but it isn’t the grid of pixels, it’s the grid of images in the state space of an overarching interactive experience.

I am intensely aware of the way my preference for Photoshop over Figma is the result of arbitrary effects due to historical circumstances, like someone preferring Bob Dylan to Kendrick Lamar. But I also feel like Photoshop imagines its user as a craftsperson and Figma imagines its user as a small business owner, so go figure.

I don’t know what the right level of zoom is for making a video game. Or for making software in general. But I loved Basic when I was just starting out. I hated Flash. I eventually learned to love JavaScript when I was using that. I know very little about GameMaker but I suspect it’s punching above its weight, given how many great games it’s produced. I assume Unreal is basically just Unity but slightly better graphics and even more annoying.

Speaking of which, look at how much this promotional image of UE5 from 2022 looks like a screenshot from The Witness…

Rest assured, I’m not planning on making my own game engine. The universe in which I make my own game engine is one in which anyone reading this post is either a decadent hedonist in a simulation where they’re a worker trapped in a slave camp or a worker trapped in a slave camp hallucinating that they are a decadent hedonist, and we’re clearly not living in that universe wink wink.

I am, however, making a new game. It’s called Q-UP. It’s a Unity game. Stand by for more details.

The final table of the WSOP main event starts tomorrow, July 15, 1:30PM Las Vegas time. I will be watching it on PokerGo.com, which, like all streaming services, is a nightmare beyond comprehension from which I cannot awake, but also fine. I think you can probably just watch it on YouTube?

This is, as these things go, a very good final table. Most notably, it features Leonor Margets, who is the first woman to reach the final table of the main event in 30 years. Margets is a Poker pro from Spain who seems super cool, and it would be amazing if she were to win it all. It’s always nice to see women succeed in Poker, there are several successful women Poker players here on Substack.

Since 1995, when Barbara Enright made the final table, around 3% - 4% of the main event entrants have been women. Given the assumption that, on average, women players have the same chance of reaching the FT as men, then these results are pretty disappointing. We should have been seeing a woman at the FT about once every 4 - 5 years. Poor results are usually a combination of running bad and playing bad. A Margets victory this week will go some ways towards updating us that it has mostly been the former. Also, LOL sample size.

Also, talent in Poker is weird, as it is in every game. We have a kind of intuitive sense that game results are a measure of skill, and, in of course, in one sense, they are. But, as measurements, they are extremely Goodhartian - they don’t simply measure the amount of some pre-existing quantity, instead they are part of a system that generates that quantity. People decide to get good at a game, and any serious competitive game is more of a measurement of maniacal, single-minded commitment to acing the test than it is a measurement of some innate ability. I suspect that a Margets win would lead to improved results for women players overall, because of what are essentially network effects - more women being inspired to play the game more seriously and push themselves harder for better results.

Speaking of maniacal, single-minded commitment, another reason this is a good final table is the presence of Michael “The Grinder” Mizrachi (second from left in the above photo) who also made the final table in 2017. It’s always fun when a well-known pro makes the FT. Again, from a statistical perspective, it’s nice to be able to point to repeated success as a demonstration of the skill-component of the game. If Poker were a game of pure chance, the odds of any individual player making it to the final table twice over that period are something like 9,000 to 1.

Also, vibes-wise, it’s a nice table, a bit less generic Poker-bro than we’ve gotten used to in the last few years. The other big story this year was Will Kassouf, the motormouth heel whose constant testing of the unwritten rules of allowed behavior made for some compelling, if extremely unpleasant viewing. Unlike Jamie King, whose speech play could be somewhat annoying but was always genuinely interesting and entertaining, or Phil Hellmuth, whose antics make him a Poker villain that everyone loves to hate, Kassouf’s behavior was unrelentingly excruciating, and a huge bummer. I think we’re lucky he didn’t make the final table, but I also hope he’s doing ok, because it genuinely seems like he’s not well. (Leave it to games, and especially Poker, to provide an experimental framework for thinking about the theoretical difference between being unwell and just being an asshole.) Anyway, it’s almost like Kassouf’s antics caused everyone else at the tournament to recognize and appreciate the value of being nice, and there’s a good sense of camaraderie among these final 9 players.

Finally, if, on either day, the action goes late, then get ready to enjoy the best thing about the WSOP, the Night Shift, featuring commentators Ali Nejad and Nick Schulman. Not only is their personal dynamic the perfect complement to the long stretches of punctuated equilibrium that is a Poker game, but Schulman’s insight into the game is a great way to get a taste of how the best players currently think, in terms of ranges, mixed strategies, and “blockers” - cards in your own hand that influence what decisions you might make in close situations. These guys are the main reason that Poker is one of the best sports on TV.

You already know what you think about Enhanced, Peter Thiel’s new steroid-friendly sports league. If you are a blue-pilled, longhouse-dwelling, future-hating weakling, you hate it. If you are a red-pilled, strengthmaxxing, future-loving Übermensch then… well actually you still probably hate it, because it’s so obviously stupid. Donkeyspace can’t help you escape this devious culture war deadlock, but we can help you try to understand it.

The premise of Enhanced is kind of beautiful in its ideological transparency. It goes like this: sport is a celebration of human performance. We have the ability to adjust and improve human performance in a variety of ways - with better nutrition, equipment, and training methods. Performance-enhancing drugs are just one more technique we can apply to enable better performance. Sure, there are safety considerations, but none that are intrinsically different from those that apply to a new kind of sneaker. By prohibiting steroids, we are intentionally forcing athletes to underperform. We are artificially holding them back, just like wokified schools hold back the best and brightest children so that the less-talented kids won’t feel bad, just like the bloated nanny-state holds back superstar entrepreneurs in fealty to some perverse vision of egalitarian collectivism. Oh! The ressentiment! The slave morality! O tempora! O mores! Don’t you want to run faster, swim harder, throw farther? Don’t you want to reach the stars?

Well, yes. But it’s not that simple!

Sports aren’t a pure expression of human physical capacity. Sports are a competitive ritual. The rules that define what is and isn’t allowed within a sport are not mere restrictions that prevent an athlete from achieving optimal performance, they are the boundaries that define the edges of the sport, creating the space within which optimal performance can exist. They are what Dave Hickey calls, in his brilliant essay The Heresy of Zone Defense, “liberating rules”. These boundaries are always, in a sense, arbitrary. They are not natural, given, pre-existing things. We create them to produce specific kinds of rituals, with specific properties, in pursuit of specific kinds of beauty and meaning.

To make the point especially obvious, consider swimming. The point of swimming is not to go as fast as fast as possible. It’s not even to go as fast as possible through the water. If you wanted to use the power of science and innovation to enhance a human’s ability to go as fast as possible through the water, you wouldn’t give them steroids, you would give them a boat. The point of swimming, as a sport, is to go as fast as possible through the water without using a boat, or flippers, or an oxygen tank, or steroids.

What Enhanced is trying to do is make a new sport that is identical to competitive swimming in every way except that it allows steroids. It’s very hard to make a new sport. If it were easy to make a new sport we would all be watching SlamBall, which is Basketball with trampolines, and is objectively awesome, unlike steroid swimming, which is just regular swimming with more liver failure.

As evidence for how hard it is to make a new sport, note that Pickleball, the world’s newest sport, was invented in 1965 and only became a big deal over the past few years. It’s also worth noting that Pickleball isn’t a harder, faster, turbo-charged, pumped-up version of tennis. It’s a different game, with a slower ball speed that gives players more reaction time, amplifying the sweet spot of cause and effect in which physics, perception, decision, and execution come together into an intoxicating stew called fun.

It’s also worth noting that you can play Pickleball and also play Tennis. Unlike steroid swimming which, if you do it, means you can’t also do regular swimming - you know, the one people watch on TV, with the medals and the records and the cheering crowds and the history books. It’s a bad idea to design a sport that is intrinsically incompatible with other activities that people might want to do, like playing other sports and having normal-sized testicles. Maybe the Enhanced folks think this will create a moat that locks players in? But you need people to care about a thing before you lock them in. You’re not supposed to build a moat around your dumb product that no one cares about yet.

There is a clear implication in this project that sports are a transaction. Enhanced will pay athletes to compete in steroidal versions of regular sports. They hope for an audience that will watch because these athletes will have slightly better physical performance as judged by some objective measure. And Enhanced will sell supplements and take a percentage of the betting action. It’s all very Rollerball dystopian. Except, again, Rollerball is awesome and this is incredibly lame.

But sports are not products, or apps, or meritocracy thought experiments. Sports are activities that people do because they’re fun. They start as competitive games that are intrinsically fun, and then evolve to become spectator events, to have institutional structures and official rulebooks and all the other trappings. Sometimes they grow big enough to have real social and cultural impact, to bend the economy, and the media landscape, and the actual, physical landscape around themselves. Big enough to have careers in, to live inside of, to give up everything for. At this scale, sports can generate deep meanings unlike any other game. But these meanings are always, ultimately, grounded in the pleasure of the core activity, which players do, voluntarily, for its own sake.

Athletes don’t want to take steroids. They take steroids because they want to win.

The wide-spread use of steroids in sports is, in part, due to a certain kind of game theoretical dilemma. Because steroid use gives you an advantage and is hard to detect, there is a temptation to do it. Every athlete knows this. Importantly, every athlete knows that every other athlete knows this. When deciding whether to dope or not, you might start out with a very slight temptation that is way below the threshold that would cause you to decide to do it. But then you think about it, and you realize that some of your competitors are going to have thresholds that are lower than yours. That makes it more likely that some are already doping. And that make your temptation to dope increase just a little bit, bringing it slightly closer to the threshold. But wait, if just thinking about it made you slightly more likely to do it, what about your competitors? They’re thinking about it too. They’re probably more likely to do it than you originally thought. Better adjust your calculations again. But hold on, that means they’re adjusting their calculations too! And so on, leading to a situation in which doping goes from being a slight temptation to almost obligatory.

If there was an easily enforceable, 100% effective, global way to prohibit steroids, athletes would gladly opt into it. This is what makes steroids different from the other sacrifices that athletes make in pursuit of excellence. They wouldn’t choose to eliminate waking up early, exercising hard, eating well, practice, study, even if they could. These things are part of the whole package, which, taken together - the pleasure and the suffering - defines the sport they love.

Sports thrive when their edges are clear, when the space of competition is well-defined, like a well-tuned scientific instrument whose results can be trusted to accurately reflect the property being measured. You can never totally eliminate the ambiguity around the edges of a sport, or any game, but we make our best efforts. Doping blurs the edges of a sport, making them hidden, ambiguous, and uncertain. Enhanced will not be able to avoid this problem. They’ll allow steroid use, but how much? On their website they claim that “every competitor will undergo rigorous, state-of-the-art medical profiling before participating in the competition.” This implies that there will be some clearly-articulated rules for what is and isn’t allowed, some level of risk or danger below which athletes must remain. But if I can gain a slight edge by pushing slightly past this level, why shouldn’t I? And why wouldn’t I expect my competitors to do the same? As soon as you draw the line, we are right back where we started, only much worse off.

The collective action trap that performance-enhancing drugs create is an example of a larger set of problems that can occur in society - Red Queen situations where players burn energy and resources but make no progress, where competition goes sideways, improving nothing and leaving everyone worse off. If Peter Thiel actually cared about the world, he could work on that.

We used to have a better class of billionaire in this country.

First of all, the origin story is awesome. Marsha Falco was studying genetics and used cards to help her think through a problem related to how traits are inherited. Each card represented a dog, and had some symbols on it, representing the various traits of that dog. And she noticed cool mathematical properties in the way these symbols overlapped. And it seemed like fun. And it was fun!

SET is also great because it’s a beautiful counterexample to a really strong theory about games - that they are made out of choices. That, as Sid Meier famously proposed, they are a series of interesting decisions. You might awkwardly append, as I often have, “...and actions!” But, even then, you’re in the same boat. Whether you like cerebral, turn-based things or experiential, flowy things, in either case it’s usually a series of things, interacting, unfolding, developing.

From this perspective, you wouldn’t expect a game that consists of pure action to be viable. Like, imagine Baseball, but instead it’s just a ball-throwing contest. For distance. Just that one thing. Or, imagine the game “How High Can You Jump” which is just that.

I realize, as I’m writing this, that I’m describing actual Olympic sports. So, obviously, games of pure action do exist. But you wouldn’t expect a card game to be one. You wouldn’t expect a completely abstract, non-physical game to be a test of one specific skill that doesn’t involve any strategic choices. And you wouldn’t expect a game like that to be deeply absorbing. But it is.

Also, by the way, if games of pure action exist, what even is game design? If “do a thing”, full stop, is a reliable way to experience the sublime, what is all this other stuff? Sometimes I think game design is about finding the interesting corners of the universe where “just do a thing” works. As Falco clearly did. SET works because of its weird and interesting features as a mathematical object. Other times I think the role of the game designer is to point at a random corner of the universe and then just keep saying “you aren’t looking hard enough.”

Anyway, games of pure action do exist, SET is one of them, and it is very beautiful. At no point, while playing SET, are you doing anything except the one action, and trying to do it better. It’s like a single note, or a single, intense flavor, where you go - huh! what’s that? And then it’s that one moment, over and over again. Like Babe Ruth at the ball-throwing contest, you just keep popping back to that same frame where you go - huh! what’s that?

Is that a game? Yes! And what a game!

The thing that gives SET that pop of huh! brainfeel is that the thing you’re doing is perceptual, it is an act of seeing, noticing, finding. But, most of the perceptual things we do are invisible to us, we’re not aware of them, we just do them. The perceptual action in SET is pitched just slightly beyond the things you do automatically.

Seeing that 3 cards make a set is sort of like seeing that something is red, but it’s got one extra twist, where you’re like - huh! ok, ok, I see where you’re going with this. It’s almost automatic but it never becomes fully internalized to where it just happens - uh, yeah, that’s red, what’s the big deal? It always feels like there’s this extra move that you can’t quite fully wrap your head around.

In a sense, seeing the solution to a Chess problem is also a perceptual action.

Finding mate in this simple position is almost just a kind of looking. Almost just a matter of noticing a fact about this particular arrangement of pieces. But it still has an element of logical operation, especially to someone like me, who doesn’t play much Chess. I still have to play out the move in my head and confirm that it works. As opposed to simply seeing it, the way that I see which piece is the Queen.

The action you do in SET, the action of seeing that 3 cards make a set, falls somewhere in between these two cognitive modes, beyond the fully-automatic, instantaneous action of seeing a color, but short of the deliberate, multi-step, if-this, then-that action of seeing a solution to a Chess problem.

The key to SET is that you’re not just doing this thing, you are trying to do it better. You are racing to do it faster and better than your opponents, or, if you play cooperatively, as I do, to beat the clock and improve on your record. So, you are made aware this action in a special way, you notice yourself doing it, and you ask yourself, “why do I suck at this?” and “how can I improve?”

It turns out these are really interesting questions. And the answers are far from obvious. Because one of the first things you notice, as you try to improve your SET skill, is that trying to improve your SET skill can make it much worse. The ability to see a set is one of those things, like throwing a baseball, or hitting a golf ball, where consciously focusing on it can really screw it up. It’s even worse, because those physical actions are complex, multi-step processes, and sometimes breaking them down, analyzing them technically, and making some conscious changes really can help you improve. And that’s not the case with seeing a set.

Getting good at SET is mostly about not stressing out, about remaining calm, getting out of your own way and letting your brain do its thing. When it works, it feels like magic, because you feel yourself getting good at a hard thing, but it’s a thing you aren’t responsible for, that you are not in charge of. The part you are in charge of is making sure that your brain has everything it needs, maybe a cup of tea, some quiet music, and then sitting quietly and not interfering.

Sometimes, when you haven’t found a set in a while, you can feel the deliberate, linguistic, logical, if-then part of yourself start to chime in - “let’s see,” you think impatiently to yourself, “we’ve got these two reds here, so maybe a third red somewhere?” Or, at an even lower level, little numerical and verbal hooks start attaching themselves to cards in your mind, and you can hear a little voice in your head counting, sorting, dividing, like the sing-song sounds of a child learning the alphabet. One, two, three. Red, blue, green.

You must never allow yourself to do this. This is the slippery slope that leads to deliberate, mechanical, operational looking, which, while tempting, is a million times slower than what you are after - the holistic gestalt of instantaneous seeing. So you hush this voice. But quietly, calmly. Because the last thing you want to do is start an argument. Learning how to silence this voice without starting an argument is the main skill of SET.

But the main reason I love SET is the particular way it felt to play, when my wife and I were playing it together on a semi-regular basis. With the iPad on the sofa between us, we played a timed version that gave you more time when you hit certain milestones of sets found, and we tried to beat our record. And doing this well meant coordinating our brains in a particular way, adapting to each other’s rhythm, each of us evolving our own style, our own specialization, without discussing it.

Or without discussing it too much. Because, oddly enough, in the two brain version of the seeing problem, the prohibition on verbalization is lifted, slightly. It’s not that we discussed strategy, we knew instinctively that that would be counterproductive. It’s more that, the chattering parts of our brains, the ones that weren’t allowed to interfere with the seeing problem, were free to talk to each other, to keep each other company. In fact, in some mysterious way this helped the other parts of our brains, the ones doing the actual work, silently, off camera, to synchronize.

So we would, occasionally, haltingly, talk to each other about the shape of the board, what it felt like, whether or not it was weird. In a kind of pidgin English that was the native tongue of the parts of our brains left over when most of our brains were quietly busy, we would discuss the personalities of the cards as they came and went, which ones were eager to join up into sets and which ones stubbornly refused - as if no one had told them the point of the game!

When one of us went on a heater (usually her) the other one would make quiet sounds of appreciation and encouragement. And, when one of us decided to hit the NO SET button (usually her) and got either the sweet chime of correctness or the harsh whinny of incorrectness, the other would make the appropriate sounds of celebration or commiseration.

And so, on many a cold night, with a fire in the fireplace, and something quiet and loopy on the bluetooth speaker (like Arthur Russell maybe, or Domenique Dumont, or some Ethiopian jazz), we would allow ourselves one more, one more? One more. And we got pretty good at doing something that can’t really be explained, something that has to do with being in the right mood without trying too hard, and surfing on the edge of yourself and another person. In a life spent playing games, it was, without a doubt, one of the most intense and rewarding and meaningful game experiences I’ve ever had.

And that’s why I love SET.

I’ve had a weird relationship with video games recently. A long time ago it became obvious to me that they were the most important and interesting domain of pop culture. After all, video games are software as an artform, interactivity for its own sake, an exploration of the things that make computation not just useful but beautiful and meaningful. Video games are amazing, both viscerally, as experience machines, and conceptually, as possibility spaces. As the intersection of games and computers, they combine our distant past - ancient rituals that pre-date civilization, with our far future - kaleidoscopic glimpses of the various directions in which our world might go.

I still feel that way about video games. I’m just not sure I like them. I mean, I like some of them fine, and a few of them I’m absolutely crazy about. I still think AAA games are something like modern-day cathedrals - sublime monuments of creative engineering, and I still think indie games have plenty of interesting experimentation, if you dig for it. But, overall, I just have this vague feeling of dissatisfaction, a general sense of malaise about video games as a… whatever it is - artform? Hobby? Lifestyle?

Look, I’ll admit upfront that this might just be me. Maybe video games are doing great. Maybe they are vibrant and healthy in all the ways you might want your artform/hobby/lifestyle to be, and maybe I’m just personally feeling burned out. I wrote a book (still available!) which was my attempt to articulate what was beautiful and important about video games and games in general. I stepped down as Chair of an academic program devoted to the creation of video games as a creative discipline. Maybe this is just me decompressing from having spent so much time and energy being an advocate for video games, being the guy that says “Hey - these things are so much cooler and more interesting then they look.” Maybe this is just a reversion to the mean.

Or maybe this is what it always looks like from the inside of an artform/hobby/lifestyle. Maybe musicians and filmmakers are exhausted and disappointed with the status quo of their domains, and look at video games the way I look at music and film, thinking “I wish we had a bit more of that energy going on.”

So, for whatever reason, I spent most of 2024 feeling a bit alienated and out of touch with video games. Then, at one point, I started watching Petscop, a YouTube liminal horror series from 2017 that takes the form of a bunch of let’s play videos of a fictional game. And this subtly inflected the YouTube algorithm in a way that it started showing me Minecraft content. And I watched some, and it really lifted my spirits!

Minecraft, with its minimalist, techno-cubist, proc-gen pastoralism, is a great-looking game. And somehow, over the past decade and a half, it has managed to keep a lot of what made it weird and interesting and beautiful to begin with. I watched the last two hours of a guy walking to the Farlands, the place where, because of its distance from the origin, the math that produces the landscape starts to break down. A trip that took him 7 years to complete.

I watched some competitive Minecraft, and was surprised to learn how much of Fortnite’s maniacal bildungskombat was in there. I watched some of the theorycrafting and server lore that populates the Minecraft content ecosystem. And I just generally enjoyed learning that this game, which I had mostly ignored, had not turned into a cesspool of software-as-a-service garbage. It made me happy that the place a generation of kids had grown up inside of seemed like an ok place to grow up inside of. That it was, like many video games, cooler and more interesting than it looked. And it made me think - chill out, video games are doing all right.

And then, because of this new found Minecraft enthusiasm, the YouTube algo served me up a video of a game I’d never heard of. A game called Fractal Block World. And then I played it, and I loved it, and it reminded me of all the things that are great about video games.

Fractal Block World was made by a small team led by developer Dan Hathaway. Hathaway was college friends with Zach Barth, and, back in 2007, worked with him on the game Infinifrag, which eventually became Infiniminer, which is the game that Notch turned into Minecraft. You can think of Fractal Block World as a kind of distant cousin of Minecraft, a splice taken from the same root and cultivated in such a way that it grew in a totally different direction - not towards accessibility and comfort and charm, but towards the thorny mindfuck aesthetics of games that explore the unconventional metaphysics of simulated space, games like Manifold Garden, Maquette, or Miegakure.

FBW is an exploration game set in a world in which space is represented fractally. You can shrink down to smaller scales, and when you do, you see that an object that was a single block when you were bigger is now made up of a complex arrangement of blocks and spaces. Continue to zoom, and you will discover that these smaller blocks are themselves not solid, they, too, are complicated 3D constellations of blocks and spaces.

Imagine looking at a sugar cube, and then shrinking down until this cube was an enormous, greebled Borg mothership, full of interlocking chambers and passageways. You thread your way through this ship, carefully avoiding guards, until you find yourself in a small room with a bunch of little objects scattered across the floor. But no, keep shrinking, and these objects reveal themselves to be vast structures, massive ziggurats, towering over a desert landscape that stretches to the horizon in every direction. Keep shrinking and the immense door of the ziggurat before you becomes a sheer cliff face that disappears into the mist far below. The cliff is riddled with openings to a twisty cave system filled with monsters and treasure. Wait, weren’t you trying to get through this door? Now you are deep inside of it. The room you entered ages ago is a fading memory, the ship an ancient myth, the sugar cube… wait - what the hell was in that sugar cube anyway?

Using this core engine, Hathaway has created an enormous, sprawling open world to explore, which you do so from a first-person perspective, in a ship with six degrees of freedom, similar to the classic 3D action game Descent. Like Descent, this world is filled with swarms of hostile enemies, and you have an array of weapons with which to defend yourself. But this isn’t an action game, exactly. It’s a game about finding your way through a strange world, understanding its logic, and diving deeper and deeper into its nested, self-similar geometry, searching for a single key buried deep within.

Visually, FBW has a radical, bare-bones simplicity that you might call “programmer art”. But FBW is programmer art in the best sense, there is no pretense or illusion, no attempt to simulate the real world, or a cinematic world, or a cartoon world. The world you are burrowing deeper and deeper into is code. You are inside of software that is unfolding in every direction around you. As a result, the world feels real. A real place made of rules and logic, pixels and math. I found myself believing in this world, and caring about it, and wanting to understand it, in a way that made me stop and think - oh, this is what people who love Dark Souls are talking about.

I experienced, while playing FBW, a kind of delirious dislocation that I hadn’t felt since first playing DOOM. There were times when I was lost, deeply lost, in a way that’s hard to describe. Not only did I not know where I was, or where I was going, I wasn’t sure what it meant to be somewhere. In FBW, space isn’t just fractal, it’s statistical. Places aren’t locations, they are definitions, rules about how things can be. The logical relationships between these rules is what creates the illusion of space that you are moving through, just like how, in the real world, the rules of physics create - wait, I’m not sure where I was going with this.

In a way, every 3D video game has this property. But in FBW it is laid bare, made real, and explored on its own terms, instead of being cloaked in the comforting and familiar illusion of realism.

FBW’s commitment to non-conventional geometry carries over to its interactive structure, the way it organizes the player’s experience through goals and mechanics. Many times, while playing FBW, I had the sense that, despite having all of this standard video game stuff - weapons and enemies and treasure and save points and power-ups - I could not use my standard video game heuristics to figure out what to do next or how to do it. More than anything else, it was this sense of unfamiliarity that I loved, this sense of not knowing, already, what I had stumbled into and how it was going to go.

This world was confusing and mysterious, but not, like in many mysterious games, because a layer of intentional obscurity had been draped over a fairly traditional structure, and not because an otherwise familiar world had been populated with neatly circumscribed pockets of mystery, little problems to overcome in the form of puzzles or combat, but because Hathaway had drilled deep into a genuinely weird and mysterious and scary corner of the universe and sent you tumbling into it. FBW is a game that starts in the Farlands and keeps walking.

Even when I was deeply lost and confused and worried I would never find my way out, it never felt like the result of sloppy or careless design. At every point, I felt like I was in good hands. I felt well taken care of. But not like I was at Disneyland, more like I was somewhere scary and cool with someone I trusted who had found something strange and beautiful and wanted me to find it too. This is outsider art, but it is well-crafted. It feels like the kind of game that was played over and over again during development. In that sense, despite being full of placeholder audiovisual assets, and despite missing the stultifying trappings of “good design”, it feels, in its own way, highly polished.

I read Edwin Abbott’s metaphysical fable Flatland at just the right age to have it blow my mind. It wasn’t just the thought experiment of picturing the geometry of space in a new way, it was also the realization of what literature could do, that it could make the strangest kinds of abstract reasoning vivid and concrete. Now, on the far side of the moebius strip of my life, FBW has hit me with a similar force.

In the late 70’s, the Polish science fiction author Stanislaw Lem complained that most SF, while purporting to offer up tales of the strange and exotic, was, in fact, in the business of domesticating the universe, re-packaging the genuine incomprehensibility of outer space and future technology and alien civilizations into comfortable, familiar tropes. I think, in a similar way, that video games have lost touch with the strangeness of software, with the weirdness of computers, the weirdness of simulating a world with math. FBW is a big dose of this weirdness, and it came along at just the right time, as something always does, to remind me what video games can do, and why I love them.

If you haven’t taken Astral Codex Ten’s AI Art Turing Test, you should. The quiz is fun, and interesting, and a good excuse to look closely at a bunch of images and think about them. The picture-making capabilities of contemporary AI continue to be a shocking fact that’s hard to wrap your head around, and whose larger repercussions will take a long time to unravel.

I like Max Read’s take on this, which is to point out that what this quiz actually measures isn’t people’s reaction to art, but people’s reaction to a series of jpegs, stripped of all context. Why does this matter? Max makes the point that the experiential qualities involved in looking at art matter, the concrete, embodied qualities of art as physical objects in actual spaces. I think this is true, but it’s one part of a larger, more general truth, which is that art isn’t just some pictures, art is a project.

This is true, in a particular way, for art of the last century or so, ie. modern and contemporary art. But it’s also true for all of the centuries before that, going back to the first prehistoric cave paintings. Art is not a single project, it’s a range of different, overlapping projects from different eras and different places, but in every era, and in every place, art is the kind of thing that some people get involved with, and pay special attention to, in a specific way. And, from this perspective, the meaning and purpose of an individual work of art is inextricably linked to its context, to the situation within which it was created, to the other works that came before, beside, and after it, and which form a larger conversation of which it is a part. This is true of 18,000-year-old bison drawings, Renaissance church frescos, Dada collages, Warhammer 40k fanart, and everything else.

To be fair, this is probably part of the reason why people who aren’t into modern and contemporary art don’t like it - “Why do I have to pay attention to a bunch of complicated contextual issues that I’m not interested in? Why can’t I just have some nice pictures I like to look at?” Which, fair enough, but that’s a very different complaint than the one implied, or made explicitly, by most of these critiques, which is that the whole thing is either a big waste of time or actively making our shared visual environment worse.

Max also makes the (related) point that it isn’t surprising that people like pictures made by AI (even when they claim otherwise) because most people have bad taste. Most of the AI-generated pictures are kitsch, and people like kitsch.

Scott Alexander has sort of seen this argument coming:

…it also seems very human to venerate sophisticated prestigious people, and to pooh-pooh anything that feels too new or low-status or too easy for ordinary people to access - without either impulse connecting with the actual content of the painting in front of you.

The implication being: “fine art” is mostly a scam. A way for elitist snobs to maintain a smug, self-congratulatory clique which presents itself as doing something profound and sacred but is, in fact, mostly just engaged in maintaining its own exclusivity through a series of codes and signals that are so arbitrary that, when push comes to shove, they themselves can’t even distinguish them from noise.

Ok. Well, he’s not wrong. Or rather, he is wrong, but he’s got a point. In fact, this is a point that has, in one way or another, been at the heart of art as a project for over a century, wrestled with by the very people who make up this clique, and for whom these codes and signals, and the profound and sacred activity they represent or simulate or obscure, have become a subject of endless, obsessive, self-critical fascination. How else do you explain Jeff Koons? How do you explain Andy Warhol?

When you look at art as a project, you recognize that Koons, and Warhol before him, and Duchamp before him, were themselves wrestling with the kinds of questions raised by this very quiz, questions about the relationship between art and jpegs, between what art purports to do and what it is actually doing, between the serious pursuit of profound and sacred truths and a speculative market in tax-avoidant ultra-luxury hyper-objects, between tacky, obscene wealth and abject, hipster coolness, between looking as optical experience and looking as social ritual, between a bunch of recursive, cerebral puzzles about the structure and limits of meaning and a bunch of pictures that may or may not make you feel a special tingle in the bathing suit area, between philosophy and decoration, between what kinds of image-making can and can’t be automated, between the irreducible particularity of the trembling human hand and the generalizing universality of formal symbol manipulation, and the capacity of either to gesture at, point to, grasp, or transmit, the infinite.

Art skepticism seems to be a common stance among a lot of rationalist and rationalist-adjacent thinkers. This general attitude ranges from Scott’s sincere attempt to carefully think through his skepticism (he followed up his AI art quiz with a post about modern architecture and a discussion of artistic taste) to Zvi Mowshowitz proudly declaring he would never set foot inside the MoMa and bluntly proclaiming that “an entire culture is being defrauded by aesthetics”.

I care about this because I like these thinkers, and I think they’re missing something important and valuable about art. I would like to be able to defend art, fine art, modern art, as a project, in terms that they would find convincing, but I haven’t figured out how to do that yet.

Perhaps, as a preliminary sketch of such a defense, I would start by calling attention to the dynamic nature of art - its necessary and unavoidable restlessness. Every work of art is both embedded within a process of perception, reaction, evaluation, and interpretation, and also an intervention into this process. Think of, at a basic level, the relationship of an artist to their audience, the artist’s desire to make something that is both genuinely new and recognizably good, the audience’s desire to see something they can understand and appreciate and, at the same time, their aversion to the formulaic, the rote, the predictable, the corny. This is the process at the heart of creativity, a process which, by its very nature, is recursive, dialogical, even, in a way, adversarial. And it is deeply relevant to a number of important issues within the general scope of the rationalist project as I see it - the reach and limits of formal systems, our ability to recognize, avoid, or extract ourselves from collective traps in behavior space, the origin and evolution of values, value drift and meta-values, coherent extrapolated volition, prediction markets, the alignment problem, all of the complicated theoretical and practical questions about how to make a loving, joyful, interesting world without using lies, and superstition, and fear. How to be embedded in a system and, at the same time, outside of it, looking in and looking out.

Here is the actual paragraph from Benjamin’s Theses on History, written in 1940:

There is a painting by Klee called Angelus Novus. An angel is depicted there who looks as though he were about to distance himself from something which he is staring at. His eyes are opened wide, his mouth stands open and his wings are outstretched. The Angel of History must look just so. His face is turned towards the past. Where we see the appearance of a chain of events, he sees one single catastrophe, which unceasingly piles rubble on top of rubble and hurls it before his feet. He would like to pause for a moment so fair, to awaken the dead and to piece together what has been smashed. But a storm is blowing from Paradise, it has caught itself up in his wings and is so strong that the Angel can no longer close them. The storm drives him irresistibly into the future, to which his back is turned, while the rubble-heap before him grows sky-high. That which we call progress, is this storm.

Later that year, on the verge of escaping to the US, he was told by the Spanish police that he would be deported back to France, and handed over to the Nazis, and he killed himself.

Whimsical, yet profound.

Max Read also points to a related recent study about AI-generated poetry. Like AI art, people can’t tell the difference, and, in a blind test, they prefer it. Here’s the AI poem that was rated most highly by the study participants:

I hear the call of nature, the rustling of the trees,
The whisper of the river, the buzzing of the bees,
The chirping of the songbirds, and the howling of the wind,
All woven into a symphony, that never seems to end.

There’s no way Scott Alexander, who is a very good writer, doesn’t recognize the badness of this poem. Even if discussing and explaining this badness requires some problematic assumptions about being interested in and paying attention to a bunch of issues related to poetry, and writing in general, as a project, as opposed to just the sorting of words into different arrangements which may or may not generate a positive emotional response.

The problem of corniness isn’t just a status game played by elitist cliques, it is also a significant, non-trivial problem. A problem about what it means to be a certain kind of self-perpetuating system, one that is both the result of a bunch of billiard balls colliding, and also, sometimes, the player who lines up the balls and decides what shot to take.

We are both the result of, and responsible for, our own taste. This is a paradox. If you pay close attention to how your own taste operates, you can sometimes catch yourself deciding to like a thing. Sometimes this is because your friends like it, or some cool person you want to impress likes it. But other times it’s because part of you, a good part, a part you trust, recognizes something in the thing, a missing piece for a new person you are in the process of becoming. You say to yourself “I want to like this thing because it is the kind of thing that the kind of person I want to be likes”. And you put yourself in the right posture to like the thing, build the necessary literacy, make the ritual gestures. But you can’t make yourself like something. Often, your existing preferences, as they already are, stubbornly refuse to budge. And sometimes they don’t.

If you pay close attention to this process, you will eventually see this as the terrain of artistic taste. Not your existing preferences, but this ongoing evolution, this back and forth, which Nietzsche called self-overcoming, but we could also maybe just call “becoming”, a process that is both deterministic and open-ended, a process that does not take place inside of you, or outside in your environment, but at the threshold between. Art is one way of paying close attention to this process. Is it a mess? Absolutely. That it works at all is a miracle.

For me, these problems, the problem of artistic taste, of corniness and kitsch, of vibe, the problem of poetry and painting, or the problem of poetry and painting as problems, the kind of difficult poetry and painting in the modern mode that Tom Wolfe hated — over-theorized, fashion-mad, intellectual style-games, wings bent backwards, caught in the howling vortex between a dead god and the atomic bomb — these problems are all deeply related to the alignment problem, writ large. The problem of where, in a material world, meaning comes from, and where it is going. The problem of computers, and software, how to use them, how to survive them. The problem of recognizing the thing that isn’t us yet. The still-unanswered questions of the 20th century, questions about rationality, civilization, and the enlightenment project. How did we fuck it up so badly? What would it look like to get it right?

Hideo Kojima is a puzzle. More than any other game designer, he represents the idea of video games as ambitious, complex, pop cultural works. A Kojima game is never an anonymous corporate product, it’s always an expression of his personal thematic obsessions and idiosyncratic stylistic and formal concerns. Kojima has earned his status as a beloved video game auteur by channeling action movie inspirations through clever and compelling game mechanics to create something totally new. His games take creative risks, explore an eccentric vision, and are genuinely weird. In an industry dominated by the predictable, the formulaic, and the generic, it’s no mystery why Kojima is considered a genius.

And yet. There’s another thing Kojima games are famous for. Interminable cutscenes filled with terrible dialogue. I mean really, really bad. Vast stretches of boring, repetitive exposition, explicating backstories that are so incomprehensibly convoluted that they wrap around into self-parody and back again. Imagine a version of The Bourne Identity where, every other scene, the action pauses for an hour and we have to listen to Matt Damon argue with GPT-2 about The Silmarillion.

This tedious exposition is interspersed with forgettable, cliché-ridden character interactions, corny jokes that fall flat, and pompous moral declarations that give the whole proceedings a wafer-thin veneer of thematic seriousness: war, when you really think about it, is pretty bad.

The good parts of Kojima’s games are truly amazing. They are pioneering explorations of a new grammar for cinematic interaction. They are visually sophisticated, experientially dense, and playfully postmodern. Even the cartoonish, over-the-top worldbuilding and plotlines might be fine, when considered separately from their flat-footed delivery. But the bad parts are truly bad.

Or are they?

Yes, they are.

But what if they’re not?

Ok, but they are.

But…

A fairly common take on Kojima is that all the things I’ve said are bad are actually not that bad. Because they are bad on purpose. Or their surface badness masks a hidden profundity. Or, while ostensibly bad, they are somehow a necessary ingredient of the overall gestalt of the work, which, because it is good, makes them good.

See, for example, this recent piece by James Bareham. He starts out making the imminently sensible observation that Death Stranding is

one of the most visually stunning, imaginative, technically accomplished, and brilliant video games I have ever played. In fact, I’d consider Death Stranding as close to a true video game masterpiece as is possible to get were it not for one significant and consistent nagging flaw that runs throughout: The English language dialogue in the cutscenes is really bad…

…a trite, nonsensical, repetitive, forced script that is so exposition-heavy that plot points are made with all the subtlety of a pickaxe smashing through concrete.

But then, eventually, he starts to change his mind…

As I progressed through the game, I became more and more impressed with the gameplay and the astonishing creative vision behind it. I adore the industrial design of the vehicles, the logo design, the design of the game’s graphic interface, and yes, I even found myself enjoying some of the dialogue in the cutscenes. Though most of the script is still bad, I now realize that it’s all part of the Kojima aesthetic and have learned to embrace it

This idea, that the idiotic cutscenes are “part of the charm” is very common. And I sort of get it. There’s a lot to love about a Kojima game, and when we love something, we shouldn’t pick it apart, take the good parts and reject the bad. When we love something, we should accept the whole package, warts and all. Maybe we shouldn’t look a gift horse in the mouth. Maybe we should just be grateful to have something this lovable and, who knows? Maybe the mysterious recipe that produced it required huge, heaping portions of inedible dreck. Art is complicated, who can say?

I can. I can say. I’m confident that these games are much worse than they could be. Worse than they should be. They should be genuine masterpieces, not half-masterpieces that require you to suffer through oceans of trite nonsense. I’m not immune to the charm of ridiculous, over-the-top genre pulp. I’m familiar with the hyper-stylized theatricality of anime. These games could be every bit as bonkers as they currently are, they could be equally eccentric and flamboyant and hilariously bizarre, and masterpieces, if they weren’t ruined by so much stupid dialogue.

Why do I care about this? Why can’t I just let people like things? Because I think that the puzzle of Hideo Kojima is, in some ways, a microcosm of the puzzle of video games in general. So many of the worst things about video games are not just reluctantly tolerated but enthusiastically embraced because, through association, they have become emblems of our beloved hobby/artform/lifestyle. The same kind of winking, tongue-in-cheek affection that people have for the “bad” parts of Kojima games reflects the way the broader video game audience has internalized their deepest flaws as being, not just acceptable, but welcome. Not just welcome, but somehow necessary. Video games are childish and vulgar and corny and silly on purpose. And we like it this way!

Except we don’t like it this way. Not really. Deep down we recognize that these things are an unfinished project, and it terrifies us. Video games are technical and creative marvels, sublime and ridiculous. They are the cathedrals of the modern world. Only we don’t know what they are for. So we tell ourselves the comforting story that they are just for puppet shows and cartoons and dressing up and make-believe and playing house and playing soldier. But they are not. They are not.

Day 2

Day 3

<attempts to make image, but cannot>

<this is the image that was downloaded onto my hard drive:>

<this is the window that opens when the [>_] link is clicked:>

In this recent thread Paolo Pedercini raises some interesting questions about my book, The Beauty of Games. After some fulsome praise that I am too modest to repeat, Paolo makes the following points:

By centering the player's pleasure, it occasionally falls into tautology: games are beautiful because they are compelling, and they are compelling because they are beautiful. Consequently, it kinda waxes poetic about gambling and doesn't spend much time on designed addiction.

The book tries to talk about ALL games, and only in positive terms, so the latter half falls in the common "games advocacy" position: all games are fundamentally beautiful and they are already making the world a better place.

The main issue with such an acritical take is that it doesn't give game makers much to think about: what we do is already awesome, we only have to convince the normies of this...

Imho a discussion on the beauty of games should also delve into how games can be ugly, evil, and bad. Maybe Frank is too kind and classy for that. Maybe that's just a position a (former) director of a game program can't afford to have.

I think this is an extremely good reading that gets at some important fundamental issues. I do worry about the book being seen as an uncritical panegyric - “yay games!” On the other hand, since I was aware of this danger and took some pains to avoid it, I want to explain my approach and see if I can defend the thinking behind it.

Let’s start with the tautology charge - games are beautiful because compelling, and compelling because beautiful. Here I think the real issue is the central framing of the book - that games are an aesthetic form - and the perspective on aesthetics, as a domain, that the book takes - which is that it doesn’t reduce. Which is to say: you can’t map the value or purpose or function of aesthetics back to some other framework in order to explain or justify it. This may seem like a cop out, but I don’t know any way around it, and in the book I attempt, at least, to acknowledge this dilemma and guide readers through it. But, yeah, since I don’t ground my appreciation of games in some other value system, the result is going to seem somewhat tautological, in the sense that it is self-contained, free-standing, independent.

What does it look like when we do try to ground games in this way? Well, we can explain them by pointing to how they evolved, and illuminating the various social and psychological incentives that motivate people to play them. We can justify them by demonstrating how they influence our behavior, how they teach and instruct, how they produce knowledge and insight. We can point to them as a source of pleasure, which is perhaps the most common way to ground them in the “ordinary” world. This is one that I try hard to avoid, despite Paolo’s claim that I “center the player’s pleasure.” It’s not uncommon to encounter, among some game designers, a pseudo-scientific approach where the ultimate purpose of games is seen as their ability to dole out pleasure-units. I can’t remember if I use the phrase “hedonic appliances” in the book to dismiss this framing of videogames. If I didn’t I should have.

One specific way of grounding games in a non-aesthetic value framework is to consider them within a context of social progress and political struggle. Given the nature of Paolo’s games I imagine this is the particular context whose absence in my approach he especially misses.

In my approach, I resist using any external value framework as an ultimate set of criteria against which to measure games’ worth - not purpose, not pleasure, not politics. This is not to say that I ignore all of the complicated ways that games interact with purpose, pleasure, and politics, the book is full of references to the connections games have with these frameworks. It’s just that I refuse to use any of them as the final arbiter, the universal yardstick, the cartesian grid against which games can be splayed out, pinned down, and evaluated.

This refusal is not an attempt to shield games from critical scrutiny. It is an attempt to make the most accurate and honest analysis of what it means for something to be an aesthetic domain. If you can’t explain games by saying that, ultimately, they allow us to learn important lessons and develop valuable behaviors. If you can’t justify them by showing how they provide pleasure or by pointing to the specific ways they improve the world, then how can you explain them at all? How can you justify them at all? And my answer is - you can’t. But that’s ok, because you don’t need to.

Explanation and justification are the alpha and omega of the real world. The ordinary world. The regular world we actually live in. And it is the nature of aesthetics to open up a space alongside that world, next to it, woven through it, in constant conversation with it, but not of it. Not contained by it. Not subservient to it. That this other world can only exist in relation to the real one, that all of its meaning and beauty and power comes from that relationship, doesn’t diminish its independent status, because, to a certain degree, the inverse is also true.

If someone made this claim about music or literature, you might nod in agreement. We are used to treating songs and stories as mysterious and sacred. But games? Games are low status, the realm of children, nerds, jocks, and gamers. Surely games need to be explained and justified. Moreover, games themselves are chock-full of explanation and justification, precise measurement and explicit evaluation. That’s the main thing we do in games - solve problems and achieve goals, sort outcomes along a spectrum of good to bad and expend energy figuring out how to navigate the space between. And you’re trying to tell me that games, despite being comprised entirely of these things, is not, ultimately, beholden to them? Yes, that’s what I’m trying to tell you. I know it’s absurd, but I also know it’s true.

Is my approach a form of “game advocacy”? Am I claiming that “all games are fundamentally beautiful and already making the world a better place”? That’s a hard question. What I’m trying to do is be accurate and truthful and describe what games are, what they do, what we do with them. Mostly, I think games are interesting. I think they are more interesting than most people realize. I want people to look at them, and see what’s interesting about them. I don’t think games make the world a better place, it’s actually much worse than that, I think games, like other artforms, make the world a place.

So, where does this leave me, vis a vis the accusation that I am soft on gambling? Am I guilty of overlooking the harms of designed addiction? I don’t know, probably? Gambling is a dangerous, potentially life-destroying force. The world is full of gambling games that want to crawl into your limbic system and drain every last ounce of life-force from your soul. From the perspective of a living soul that wants to hang onto its life-force, this is a BAD THING. But the world is also full of anti-gambling counter-measures - warnings and customs and edicts and norms that want to protect your soul and keep it safe. I don’t want to be a combatant in this conflict. I want to pay attention to gambling. I want to see it, and understand it, as a thing in the world, a thing that is capable of creating beauty and meaning, as well as destroying it.

I think most people should probably just avoid gambling games. Most people should avoid hypnotic, compulsive, addictive games in general. But I also think that, if you really want to understand games, you should learn how to play these games. Which means learning how to stop playing them. You should learn how to disappear into them, how to let them swallow you whole, and then learn how to extract yourself, safely, on the other side. This is a skill you can develop, and getting good at it will not only help you safeguard your soul, it will also teach you some things about how your soul is constructed in the first place.

Is my approach “acritical”? I don’t think so. Aesthetic forms are, more than anything, realms of critical judgement, in which we seek to make fine-grained distinctions between experiences, to compare and contrast them, to articulate how and why some are better than others, to discover what we want from them, and why; a process that unfolds in the intersection between our private experiences and and our shared lives. But doing this well requires perceiving them clearly. I think that it is difficult to see games clearly. Partly because they’re complicated, multi-dimensional, experience-generating state machines, and partly because some of the common tools we use to see them are crude, and obscure and distort as much as they reveal.

What may appear, in my approach, to be a cavalier disregard for the important work of critical judgment is, instead, an attempt to set aside these common ways of looking at games, the ones that come easily to mind, that come so easily to mind we don’t even notice them. To set them aside so that we can see what we are doing, really see the games we are playing, and begin the serious work of critical judgment in earnest.

In my house, every evening, before going to bed, we engage in a strange ritual. We watch two men play a game we barely understand. Then we look at each other and shake our heads - what was that? Sometimes we point at the TV and make noises, like the monkeys shrieking at the monolith at the beginning of 2001: A Space Odyssey.

The two men are Rob Barrington and Gavin Wolpert. And the game is Bridge.

I like complicated games. I especially like the experience of watching other people play them, in which the effort you expend trying to understand what’s going on allows you to appreciate, vicariously, what’s happening in the mind of the player, like the toy steering wheel that lets Maggie Simpson pretend she’s driving the car.

I like finicky, highly-technical M:tG control decks, I like watching people try to pull off a clandestine finesse in Hanabi, I like hearing Levy Rozman rattle off opening variations as if they were 5-dimensional origami instructions, I like it when console gamers get out the MS Paint, I like when players build computers inside a game that can run the game they are inside of, I enjoy thinking about the Pretzel Conundrum.

And I’m here to tell you, none of that stuff can hold a candle to Bridge. This is the real deal. Bridge is so galaxy-brained deep, so scarily complex, it makes every other game look like Uno. And yet, for the most part, no one seems aware of this fact. Bridge, having been super popular approximately one lifetime ago, is now in a kind of cultural trough, embedded in a constellation of signifiers that make it a weapons-grade antimeme. But not for long. Mark my words: Bridge will return. It is too good. Too weirdly, uniquely beautiful. And when it does return, when you and your hipster friends start arguing about five-card majors over your bottomless mimosas, the people who run my estate will link to this post and say: See? He told you!

So what is it? What makes Bridge so amazing? Well, the thing that makes it amazing is also the thing that makes it difficult to learn and hard to understand - Bridge, Real Bridge, Contract Bridge, the game that Rob and Gavin play, isn’t just a game, it’s a nested stack of tightly interconnected metagames.

On level zero of the stack you have the basic structure of a trick-taking game. You know the drill - someone leads a card, everyone else has to follow suit, the person who played the highest card wins the trick, and gets to lead the next trick. This is the logic puzzle at the heart of the game, and everything else is built on top of this foundation. Bridge is a partnership game, so solving the trick-taking puzzle involves weaving back and forth between your and your partner’s hands, while dodging the cards in your opponents’ hands. Ok, cool.

Things start to get interesting during the auction. In the auction, we take turns declaring how many tricks we think we will be able to take, given that a particular suit is “trump” (wins even if it wasn’t led). The auction is explicitly “meta” in the same way as betting in Poker and the doubling cube in Backgammon, it’s a kind of thermodynamic trick for taking a game with randomness and squeezing order out of it. You are presented with the game’s core logic puzzle in a randomized state and asked, not just to find the best move, but to predict the various ways the puzzle could unfold, to model the whole puzzle in your head, to picture it as a series of phantom branches forking off into the future, and make a strategic evaluation of that overall picture.

A game’s core logic puzzle has to have the right level of complexity to support this kind of metaplay. The core logic puzzle in Chess is too complicated to be folded back on itself in this way. And so Chess, as beautiful as it is, is really just a big puzzle. Bridge is a strategy game in which Chess-like logic puzzles are the pieces.

But we’re just getting started. Because, remember, Bridge is a partnership game, so during the auction, we aren’t just declaring how many tricks we think we will be able to take, we are communicating to our partner (and, unfortunately, our opponents) about the content of our hand. In the Bridge auction, the series of escalating bids is a tiny language in which each statement is both a move in the game - a binding commitment to a particular arrangement of specific game variables - and, at the same time, a comment about the game - a way of coordinating our actions with our partner to find out which arrangement of game variables would be optimal for us.

Every bid in a game of Bridge is overloaded with this double-meaning. In linguistic terms, every bid in Bridge is a speech act with concrete effects (like saying “I do” during a wedding ceremony or uttering the special word that triggers a magic spell) and a signal by which we mean to send information from a speaker to a listener. It is both of these at the same time. More than anything else, Bridge is about this double-meaning. It is about the intricate dance of information and action by which we say things about the world and do things to it.

This is the source of the game’s turbulent, never-resolved tension between the natural and the artificial. Sometimes you say “2 clubs” because you have a bunch of clubs, and sometimes you say it as part of a pre-arranged code that has nothing to do with how many clubs you have. Serious Bridge is built around these codes, every partnership uses an elaborate bidding system that allows them to signal in this way. And they’re public knowledge, by the way. You aren’t allowed to have a secret system.

So far so good. But everything I’ve described so far is true of Bridge as it already existed up to the mid-1920’s, which is when it got really good.

In 1925 beloved yachtsman and all-around rich guy Harold Stirling Vanderbilt released a scoring system for Bridge that transformed what was already a fantastic and very popular game (Auction Bridge) into a brilliant masterpiece that became a world-conquering pop cultural phenomenon (Contract Bridge).

What did he do? Well, we are rapidly entering a zone beyond my pay grade. But, as I understand it, the scoring system Mike came up with (yeah, his nickname was Mike) made bidding matter. It amplified the consequences of bidding so that you were rewarded for bidding in the most precisely accurate way and punished for deviations from this accuracy. This incentivized a greater focus on the load-bearing capacity of bidding as a communications channel and encouraged players to explore its potential as deeply as possible, leading to the complex bidding systems that we have today, and the rich strategic space that goes along with them.

(I always think of Mike Vanderbilt alongside Lizzie Magie and Alfred Butts as one of the great unsung heroes of early 20th-century game design. Before it was a profession, game design was done by inventors, tinkerers, radicals, and dilettantes - three cheers for these beautiful weirdos!)

If the auction folded Bridge over itself, to squeeze out more structure, more pattern, more information, the scoring tweaks that created Contract Bridge folded the game again, doubling the significance of every detail of player choice and action. And the competitive format that evolved around this game became yet another twist designed to wring out the last remnants of noise from the randomness of the shuffled cards. The highest floor in the metagame stack - Duplicate.

In Duplicate Bridge everyone gets the same cards, and your actual score is the difference between how many points you made and how many points all the other players made in the same situation. It’s as if Bridge players looked at Poker, in which the cold white peaks of good decision-making gradually emerge from the uncertain mist of random variance over a long, slow run of thousands of hands, and said “We don’t have time for that. We’re going to die soon!” So they made the long run happen all at once, in a single afternoon, by giving every hand, every random distribution of cards, lucky or unlucky, to a hundred different teams at the same time, and stack ranking the results. A duplicate Bridge tournament is sort of like a computer program for determining the best way to play a set of hands, in real time.

Which brings us back to Rob and Gavin. The format that these guys play, the one we watch every night, is a special kind of duplicate tournament. Now, you might have thought that we’ve stretched and folded this taffy as much as possible, but there’s one more twist coming - bots. In this format, every player is an individual who plays with a bot partner against two bot opponents. All of the bots are the same algorithm, using the same bidding system. On the one hand, this reduces variance even further, squeezing yet more randomness out of the system. On the other hand, it adds an additional layer of technical skill to master - knowing exactly how the bots “think”.

Bots! As if Bridge, this twisted knot of predictive modeling, protocol hacking, and information theoretical meta-puzzles, needed another cyberpunk ingredient to make it the most 2024 possible game. Now it has bots in it!

Bridge is a game that is breathtakingly dense. We tried learning it a few years ago, back when we were young enough that it was “cute” that we were into it, but it was just too hard for us. And now, even if we did know it, we would just be two old people who are bad at Bridge. But no one can stop us from watching it on YouTube.

Every hand opens the same way, with a glimpse of the familiar puzzle that starts with 13 cards. We count the points - 4, 8, 11, 14, 15 - this much we remember, but already Gavin and Rob are miles ahead. Rob with his FM-smooth drive time baritone, and Gavin stopping and starting like a free-jazz soloist chasing a fleeing melody. Back and forth they speculate about the distribution of the unseen cards based on the bids being made, hypothesize about most- and least- likely universes, spin out scenarios and counter-scenarios, race like wildfire through thickets of logical deductions that would have taken us hours to navigate, and then stop dead to ruminate in excruciating detail on a decision we wouldn’t have seen at all, whose significance we can barely fathom.

Sometimes, because of the tournament structure, they’ll choose to take a line of play that is less likely to lead to the best outcome, because, in order to win, they need to do something different than what most other players are likely to do. Which is, of course, a beautiful example of donkeyspace in action.

Once, about a lifetime ago, America was obsessed with a dense, complex, abstract strategy game, obsessed. This unforgiving, cerebral, hard-as-nails game was more popular than Wordle, more popular than Pickleball, more popular than Minecraft and Among Us and Fortnite combined. I find this fact amazing and inspiring. There is an appetite for such things. Under the right conditions, we will decide this is what we want to play - a game that invites us to bend and stretch our minds into surprising new shapes as we try to solve the hardest problems we can think of, together.

If there’s one thing we know for certain that AI can do, it’s play games. Before computers were even invented, there existed game-playing software. Not only can AI play games, they are extremely good at it, much better than us. In fact, games, these miniature worlds with precisely-specified features, are the quintessential domain for AI. Games are AI’s natural habitat. Following rules in pursuit of some arbitrary goal is what AIs are all about. To the degree that they can accomplish any activity — fold proteins, write poems, make pictures — they do so by turning that activity into a kind of game. It’s almost as if games have always had AI coiled up inside them, waiting to get out.

And yet, recently, I can’t shake the feeling that AI can’t play games. That no existing AI is capable of playing a game. Not Deep Blue, not AphaGo, not Stockfish or TDGammon, or Chinook. Where did this crazy idea come from, and what does it mean?

I blame Jim Rutt. It was his little game Network Wars, which I’ve been playing obsessively for over a year, that got me started on this. I wrote about this game before, and specifically about this topic: about how the game’s simple AI “opponents” are better understood as rules or features of the game, not players. But since then, I haven’t been able to get this idea out of my system, it has continued to grow, and now seems to me to be a kind of skeleton key for understanding AI in general. Let me see if I can give you a glimpse of what this idea feels like.

Is Bowser playing Super Mario Bros? No, of course not, what an absurd question. Super Mario is a single-player game, and Bowser is just an element within that game — a subroutine, a set of rules. It may sort of look like Bowser is your opponent, it may look like he and Mario are two contestants engaged in a contest, but this is just a wafer-thin representational veneer applied to the surface of the game. To ask if Bowser, or, for that matter, Mario, is playing a game is to make a fundamental conceptual error, like asking if Hamlet is acting. Are dice playing Monopoly? Is the net playing Tennis?

Ok, let’s try a harder one. Is Emperor Arcturus Mengsk playing StarCraft 2: Wings of Liberty? (Mengsk is the commander of the enemy forces the player opposes in some of the missions in the single-player campaign.) Intuitively, this seems like almost exactly the same question as before. There is no Mengsk. Like Bowser, he’s just a cartoon added to the game to give thematic “juice” to some of its rules and subroutines. Asking if Mengsk is playing StarCraft is like asking if gravity is playing Tetris.

Ok, now how about this one? What if, in StarCraft 2, the player chooses “Play vs AI” and then selects the “Medium Difficulty AI”. Is that AI playing StarCraft? Ok, now we’re talking. Intuitively, yes, of course it is!

Now what if I told you that StarCraft’s Medium AI is identical to Mengsk? What if it was the same exact code? (This is not, to my knowledge, the case, but for the sake of argument, imagine it was.) Do our intuitions change? And in which direction? Do we now think Mengsk is playing StarCraft? Or do we think the Medium AI is not? Or do we still think one is playing and the other isn’t? And if so, why? Is it simply a matter of framing? Of context and interface? At any given moment, the exact same thing is happening, You’re selecting your units and giving them commands, and the software is responding by selecting their units and giving them commands. If I didn’t specifically tell you which menu button you clicked to initiate this battle there would be no way to tell the difference. Is there really a difference?

Let’s consider a different example. Take a card on which is printed instructions for the optimal way to play Blackjack. Is this card playing Blackjack? Obviously not. What about a person who is using this card and following these instructions, are they playing Blackjack? I would say yes, certainly they are. So what’s the difference?

Optimal Blackjack Instructions are a strategy within the game of Blackjack, a complete description of which moves to make under which conditions. It doesn’t make sense to say that a strategy plays a game. Players use strategies to play a game. They pick strategies, they develop strategies, they apply strategies, they follow strategies. But strategies themselves don’t play games, that much seems pretty straightforward.

What about a Blackjack dealer? Is she playing Blackjack? Intuitively, I would say no, she seems a lot like Bowser, more like a mechanism within the game than a player. But isn’t the dealer executing a strategy, just like the player using the Blackjack strategy card? Not really, no. If the dealer deviated from her pre-determined strategy, if, for example, due to a lapse of attention or a capricious whim, she neglected to stand on 17, the actual players could stop the game, point out the error, claim that the game had broken and demand a re-set. Just as if the wind blew down the net in a game of Tennis. The human dealer is really just a method for implementing the rules of the game. The rules are arranged to make the dealer look like a player - they “have” cards, they “take” actions, but really there’s no player there, just rules. Just like Bowser, Mengsk, and the AIs in Network Wars.

What about the casino? Is the casino playing Blackjack? You aren’t really playing against the dealer, but are you playing against the house? Huh. Yeah, strangely enough, it kinda seems like the casino is playing Blackjack to me. Not the building, obviously, but the… the corporate entity I guess. How could that be? How could this arrangement of people and rules and norms and laws and equipment and software and whatever else comprises a casino play a game? Well, for one thing, unlike dealers, nets, and Bowsers, the casino chooses to play. The casino voluntarily offers the game to the other players, and can quit playing when it wants to. Also, the casino picks a strategy — what stakes to accept, when to shuffle, how many decks to put in a shoe. And these things work together. The casino is aware that it is playing a game. It sees the game. Decides to enter into. Notices the results. Can change its strategy.

The Blackjack strategy card sort of does some of these things. It can, in a crude sense, “see the difference” between the dealer showing a 6 or a 7, and “choose” whether to stand or hit. But, even if you grant this generous animism, the card can never “see” the game. In order to see a thing you have to be able to pick it out from the background. You have to be able to distinguish between it and other things. The card can (sort of) distinguish between 6s and 7s and between Ss and Hs. And that allows it, in a sense, to “move through” the world of Blackjack. So let’s be wildly imaginative and picture this thing as a tiny creature skittering through Blackjack space. Even then, it could never distinguish between Blackjack and Gin Rummy. How could it? Even our crazy animist fantasy version of this thing is fully contained inside of Blackjack. How could it ever see it?1 Maybe you could make one that did. You’d need a very large card.

Let’s consider some other test cases. What about a little kid who barely understands the rules? Are they playing a game? I would say yes. In fact, I would say a little kid who barely understands the rules is, in some ways, the quintessential game player. (Sometimes the best player in the world at a game doesn’t know the rules.) What about animals? Is a dog playing fetch playing the game of Fetch? Again, I would say yes.

It’s also possible to think of lots of ambiguous cases, grey areas where something is sort of a game, or where someone is sort of playing. Is the Michael Douglas character playing the game in David Fincher’s The Game? If you force someone to play a game against their will are they playing? What counts as sufficient force? Whining? Cajoling? We’ve all been at card parties where someone who doesn’t like Bridge was strong-armed into being a reluctant fourth,2 were they playing the game?

So we have some examples of things that seem sort of like they might be playing a game but which, on closer examination, are clearly not. They are rules, mechanics, features, strategies. In contrast, we can think of examples of things that clearly are playing games - little kids, animals, casinos. As well as border cases where it’s just not clear.

So where does AlphaGo fit in this menagerie? Moreover, why does any of this matter? Is this just semantics? Just definitional nitpicking? Or, worse yet, is it AI-skeptic goalpost moving? One of those situations where, as soon as AI does a thing, we decide it’s not really a thing, or the AI is not really doing it, or not doing it in the way we do it, and therefore it doesn’t really count. One of my game designer friends, when I told him my suspicion that AlphaGo wasn’t playing Go any more than Bowser was playing Super Mario World responded:

or any more than an airplane is actually flying, given that it doesn’t flap its wings. Deep Blue was an amazing and brilliant Chess player, full stop. Because we only have one way to actually measure the brilliance of chess players.

But I think it does matter. And here’s why. Because AlphaGo is broken. I, a mediocre amateur Go player, can beat AlphaGo. OK, well, technically, I can’t even actually play against AlphaGo, but I can beat KataGo, as demonstrated by the brilliant work of Tony Wang et al in Adversarial Policies Beat Superhuman Go AIs which I’ve written about before. KataGo, like all the other top Go AIs, is based on AlphaZero, AlphaGo’s superior successor, and they’re all broken in this way.

And the way modern Go AI is broken is fundamentally related to this exact issue. The reason that you and I can beat superhuman Go AIs with a stupid trick is that they are stuck inside the game. They can’t see the game. They can’t see that they are losing, over and over again, to a moron. Because, unlike my 3-year old granddaughter, and my cat Duke (RIP), and Michael Douglas, and The Golden Nugget, they have no concept of what a game is, what winning and losing is. They are a complete and thorough game-playing process, a complete set of instructions for how to play, for which moves to make under which conditions. But this just makes them a very complex strategy, able to see moves with superhuman acuity, but utterly incapable of seeing the game itself.

Furthermore, this is a problem that’s easy to fix. The KataGo team has probably already patched, or is in the process of patching, the code so that the specific dumb trick of this particular adversarial strategy no longer works against it. And that’s the fix. No, not the patch, but the overall process of Team KataGo noticing that the AI is losing, and caring that it’s losing, and deciding to do something different. This is what playing a game looks like!

So that’s what makes this philosophical puzzle more than just a curiosity. Because it’s an engineering problem. Making AI systems robust to adversarial policies is a difficult and important problem. A problem whose solution may well require figuring out how to make AI systems capable of seeing the game they are playing, of noticing, caring, and deciding. And it’s not just AI safety. The reason AI sucks at poetry is also fundamentally related to this same problem. It’s my view that a version of this problem is the reason modern AI, despite being amazing at so many tasks, also sort of sucks at all of them.

I’m not saying that making an AI that can play a game means solving the hard problem of consciousness. But I’m not exactly not saying that either. Maybe consciousness was just evolution’s way of solving the engineering problem of how to prevent intelligence from getting stuck inside of games. I don’t know. But I do know that, like it or not, doing practical AI engineering in 2024 means wrestling with some thorny philosophical puzzles.3

So, there you have it. That’s my theory of why current AI can’t play games, and how it could. I’m not like John Searle, making some grand impossibility claim. I think it’s a tractable problem, but I do think it means making some fundamental changes to our approach.

And, unlike my friend, I think we have more than one way to measure the brilliance of Chess players. For example, Magnus Carlsen, who is not only the greatest living Chess player, but quite possibly the greatest to have ever lived, is bored of Chess. “I find classical Chess stressful and boring.”

That’s what it looks like to not be trapped inside of Chess. That’s how to play a game.

I just got back from the Game Developers Conference, where one of the big topics was “Are you playing Balatro?” The game is a rogue-like (because you play it over and over again) deckbuilder (because you modify a deck of cards over the course of each game) that combines a standard 52-card deck and poker hand rankings with a bunch of extra stuff that interacts in various crazy ways. It’s a fundamentally simple game with a ton of charm and polish in the execution, and it’s a lot of fun. But what makes it especially interesting to me is the way it combines skill and luck, specifically, the way it has a whole lot of both.

The relationship between luck and skill is a somewhat under-theorized aspect of games. It’s easy to think about games like Chess, where the outcome is completely determined by the strategic decisions of the players, and pure gambling games, where the outcome is completely random. But what happens when, as in so many games, strategic decision-making and randomness intermingle?

There’s a nice chapter on this topic in Characteristics of Games and Greg Costikyan touches on the subject in Uncertainty in Games, but both of these treatments are fairly basic. One useful idea is the distinction between input randomness (random elements that the player responds to by choosing an action) and output randomness (when randomness influences the result of a player’s action), a concept originally developed by Geoff Engelstein on his podcast, although I first learned about it from Keith Burgun. But, overall, the interaction of luck and skill in games remains somewhat mysterious. Balatro is a good opportunity to experience that mystery directly, and an invitation to consider what it means.

Although it borrows the hand-rankings from Poker as the foundation of its scoring system, Balatro doesn’t really resemble Poker that much. At its heart, it’s what I call a shopping game. Over the course of each run, you are offered different game-modifying Joker cards to acquire. Jokers combine in various ways to create powerful point-scoring engines. Assembling these engines and tweaking your deck to work with them is the core of the gameplay.

But the thing that Balatro does share with Poker is that it is both deeply strategic, with lots of opportunities, and incentive, to calculate the effects of the actions you choose, and high-variance, meaning the outcome of any one individual game session is only loosely correlated to the strategic choices you made.

I love games like this. Why? I’m not exactly sure. Partly its just the high-octane cocktail of two complementary emotional ingredients - the degenerate gambler’s love of intoxicating action mixed with the pretentious intellectual’s love of sober problem-solving. But I also love the fact that games like this are confusing, misleading, hard to analyze. I’ll never forget the feeling I had when I first realized that you could study Poker the way you studied Chess. Sneaky! I thought. A game that is less dumb than it looks. (This is true, to a degree, of all games.)

But the confusion goes both ways. Over the past few weeks I’ve spent a considerable amount of energy arguing with one of my game developer friends about how to interpret Balatro’s luck/skill dynamic — the degree to which our results in the game reflected the quality of the strategic choices we were making.

“It’s a slot machine” I said.

My friend countered by pointing out all the different ways in which your strategic choices could influence your odds of winning, and gestured to the fact that our results were improving over time as evidence that strategy had a significant impact on outcome.

It’s not that he was completely wrong. But the thing I wanted him to understand was the fundamental principle of high-variance games, which is that they drastically limit our ability to interpret the results of any single run as evidence of the quality of the strategy we are using. Take any strategy, X. (A strategy, in this sense, means a complete description of what actions to take under what circumstances. So, for Balatro, not just using a certain Joker or buying a certain kind of card, but which Jokers and cards to buy under which specific conditions.) Run 100 sessions of the game using strategy X and plot the results. You will see a broad distribution of results, clustered around some point but falling off in both directions. Taking a random sample from that distribution gives you next to no information about the shape of the overall curve. It’s not exactly zero, but it’s practically zero.

This is the way in which Balatro and Poker are similar. In Poker, a common mistake for beginning players is to win a hand, or have a winning session, and think “Ah! This new approach I’m using is working well!” This mistake is referred to as being results oriented. It doesn’t mean that Poker isn’t deeply strategic, it just means that that’s not how you evaluate the quality of different strategies in Poker. In high-variance games like Poker and Balatro, you evaluate strategies by theory-crafting — analyzing the game with math and logic, and developing a complex model of the game to understand its important strategic principles. Over time, you can look at the statistical shape of your long-term results as (fuzzy) evidence for how good your strategic principles are, but here’s the important thing: your key insights must come from taking the game apart and understanding how it works, not by scrutinizing the signal of your immediate results.

Why does this matter? Why was I so adamant about insisting on this aspect of Balatro? It’s a fun game, a bit silly, who cares?

Because slot machines are dangerous. It’s fun to play with dangerous things, but you can blow your head off with them. For me, a big part of the fun of playing with slot machines is knowing how they work, and a big part of how they work is by manipulating the cause and effect mechanism in your brain in exactly this way, by hijacking your brain’s reward function and giving you a false impression about the amount of agency you are exercising. Our brains are hyper-tuned to see signals in our environment, to interpret them as expressions of an underlying structure. Slot machines are creatures that have evolved to feed on this behavior by generating a signal that sounds like the message “keep digging” but is really only a complicated kind of noise. You can eat them, and they are delicious, but only if properly prepared. Which means, first and foremost, recognizing the trick and not falling for it.

It’s good to see the Chess in Poker, and realize how that game is smarter than it looks. But it’s also important to see the slot machine in Balatro, and realize the ways in which it is dumber than it looks. Again, this is not to deny that Balatro is, unlike actual slot machines, full of complex, interesting decision-making. If it wasn’t I wouldn’t still be playing it. I still often find myself in situations where I’m not sure what to do, and by carefully thinking through these situations I can feel myself learning, developing a better understanding of the game, and improving my overall strategy. But there’s a sense in which this process is happening despite the game, in a kind of wrestling match with the game’s explicit reward structures, the way it doles out wins and losses and big scores.

You can see this in some of Balatro’s specific structural features:

• The game is organized as a series of scoring challenges. In a good run you will often be scoring far more than the necessary targets, but the game doesn’t pay attention to this excess score, it doesn’t affect the outcome at all. As a result, you often find yourself continuing to tinker with your deck, making it more efficient and improving its scoring power for the fun of it, in a way that the game is basically ignoring.

• Then, every time you “win” a game by beating the final scoring challenge, the game invites you to continue on to “Endless Mode” (really, just a series of additional, rapidly-scaling scoring challenges.) This moment is deeply confusing. Wait - what was I doing? Was I trying to get a win or was I trying to see how high I could score in Endless Mode? In reality, the best strategy for maximizing your chances of winning are different from the best strategy for maximizing your chances of going deep in Endless. The former requires adapting, scrappily, to whatever combo pieces the game offers you, the latter requires taking greater risks to seek out the smaller set of overpowered combo pieces that are capable of producing astronomical scores. In fact, it’s already known how to “beat” endless mode by generating more points than the game’s scoring function can handle, it just requires waiting patiently for the necessary ingredients to show up. By blurring the difference between these contrasting goals, Balatro amplifies the sneaky disconnect between player action and game result.

• Meanwhile, win rate, the one thing that all of the player’s clever strategies and complex problem-solving is optimizing for, is completely ignored. The game doesn’t track it at all.

• Once you get into the game’s higher difficulty levels you quickly find yourself “start scumming” — restarting a run as soon as the first few levels go poorly. Yes, you could try hard to play optimally in order to eke out a few extra levels in these unlucky runs, hoping to encounter a viable combo piece, just like you can try hard to optimize the performance of your deck in a lucky run where you’ve already guaranteed victory half-way through. In both cases the game isn’t paying attention at all.

One of the reasons I love games like this is that, unlike Chess, this is a wicked problem. What’s the best way for an agent to extract information from a noisy environment? We make sense of the world through stories of cause and effect, but, on close examination these stories are far less simple than they appear. In The Book of Why, computer scientist Judea Pearl identifies three levels of causal inference:

1. Association. Discovering, through observation, the connections between phenomena.

2. Intervention. Predicting and testing the effect of our own actions on the phenomena we observe.

3. Imagination. Considering counterfactuals, hypothesizing about what might happen if things were different than they actually are.

In a game like Balatro we encounter an even harder version of this problem: how do you learn in an environment that isn’t just chaotically noisy but that is loaded with adversarial signals that have evolved to mislead you?

The real world is full of examples of people wrestling with versions of this problem. In a recent column, Donkeyspace hero Matt Levine described Elon Musk’s particular version of the struggle:

The Elon Musk theory of corporate governance is that everything that Elon Musk does is good for his investors:

1. Elon Musk has made many hundreds of billions of dollars for investors in his various companies.

2. Those investor returns come, in large part, from Musk’s personal attributes and efforts: Those companies would not be successful without his perfectionism, drive, management style, techno-optimism, etc.; nobody else could have built those companies, and even today they would not perform nearly as well without Musk’s intense personal involvement.

3. Those personal attributes, which are good for investors, are inseparable from the rest of his personality, which is … oh, you know. “I reinvented electric cars and I’m sending people to Mars in a rocket ship,” Musk once said on Saturday Night Live; “did you think I was also going to be a chill, normal dude?” His personality only works as a complete package; you can’t get the shareholder value creation without the bad tweets.

4. Therefore, the bad tweets are also good, because they enable the good stuff; they are the compost that feeds the flowers of shareholder value.

This theory is … I mean, it’s probably not wrong, is it? It is pretty unhelpful as a theory, though. It proves too much. It implies that any conflict of interest between Musk and his shareholders is impossible, because whatever Musk wants is necessarily good for shareholders — even if it is bad for shareholders, it is good, because the bad thing is just the price they pay for Musk’s goodness

Anyway, here is Elon Musk on the Elon Musk theory of corporate governance:

Elon Musk said that his prescribed use of ketamine alleviates periods of low mood and is in the best interest of investors in Tesla Inc. and the other companies he runs.

For Wall Street, “what matters is execution,” Musk said in an interview with former CNN anchor Don Lemon streamed Monday on YouTube. “From an investor standpoint, if there is something I’m taking, I should keep taking it,” he said referring to Tesla’s success.

Musk said he takes the drug as prescribed periodically to treat what he described as “chemical tides” that lead to depression-like symptoms. 

I am sure Elon Musk has good doctors, and I am not going to second-guess the medical advice he is getting, but the corporate governance theory here is “if there’s something I’m taking, I should keep taking it.” That logic applies to taking ketamine on medical advice, or taking cocaine against medical advice, or anything else. Whatever he does is necessarily good.

Each of us, in our own way, is in the same boat: trying, and often failing, to understand the connections between the things we do and the results that follow. Balatro is a laboratory for experimenting with this problem, a laboratory with blinking lights, soft carpets, and free drinks.

The reason gambling is fun is that we are not our reward function. Maybe once we were, maybe once, long ago we were nothing but a little knot of signals that was able to remain stable far out of equilibrium with its environment, a little loop of Bayesian inference that was able to climb the signal gradient towards more and more complex and robust forms of stability. But now that we know that, now that we can look back and see that, we are no longer fully embedded in our reward function, it has become something we can use, something we can play with, instead of simply who we are. With some effort we can partially extract ourselves from the loom of our reward function, look around, and ask: “Where am I?” and “Where do I want to go?”

For a while I’ve been playing (but mostly watching) GeoGuessr, the game where you are shown a random image from Google Street View and try to identify its location. Last week I learned that GeoGuessr has been given the full eSports treatment. I watched the Americas Division Finals of the GeoGuessr World Cup. And let me tell you people, I’m a believer.

I love when games are too simple to work, but work anyway, and GeoGuessr is fun in exactly this surprising way. It’s a game of pure skill, there’s very little in the way of player interaction or strategic decision making. Although, as we know, sometimes when you have very little of a thing you end up magnifying its influence, and different players’ move/search and clock-management strategies are an exciting aspect of the competition. But for the most part it is, like Golf, more of a side-by-side comparison than a head-to-head confrontation.

Like SET, GeoGuessr is a game about perception, about zooming into the first step of the OODA Loop and discovering that there is already another tiny loop inside there, waiting to take you for a ride. GeoGuessr is about quickly extracting information from data in this one, precisely-defined domain, and a big part of the fun is seeing how good the players are at it. They can glance at what looks to me like a generic, non-descript scene — a stretch of road, a distant hill, a few scrubby trees — and immediately pinpoint its location. It looks like magic. Watching people do impossible things with miraculous ease is a big part of all sports. Here what they’re doing is strangely familiar (looking at pictures on the internet, just like us) and deeply primal (looking at the world and trying to make sense of it, as we’ve always done).

It looks like magic, but if you watch a good player stream the game while talking through their thought process you can learn about all of the subtle vectors of meaning that feed into their uncanny intuition — the way different parts of the world vary in road markings, vegetation, signage, terrain, and architecture. When talking about these clues, players use the gamer term for the current consensus regarding a game’s optimal strategies: “meta”. GeoGuessr has “car meta”, “bollard meta”, “pole meta”, and so on, each one it’s own little dense pocket of Pokémon knowledge hidden behind the mundane surface of the world, waiting to be studied and applied.

Like Jeopardy, GeoGuessr is a game where, while watching, it’s natural, effortless, almost unavoidable to play along. When the image pops up we can’t help but try to place it, and sometimes we’re right! And, as we encounter example after example, and absorb the techniques of the expert players we are watching, our results gradually improve. This is true, to a degree, for all games. But it is far more true of GeoGuessr than, for example, League of Legends or Valorant, because the thing we’re doing when we mentally play along with GeoGuessr is actually playing the actual game.

One of the things I love about GeoGuessr is the images themselves. There’s something incredibly beautiful about seeing the world this way, as a series of random snapshots of ordinary places. Not the Eiffel Tower or the Taj Mahal or Big Ben, but a quiet suburban street in a Paris suburb, a pile of bricks next to a barbed wire fence on the outskirts of Uttar Pradesh, a cellphone repair shop in Tourquay. Each place stubbornly full of little details that make it unique. The world is so immense. You could study the world for a million years and never learn all its secrets. Just like Chess.

“But Frank,” I hear you say, “what does all this have to do with AI? Aren’t you contractually obligated to make everything about AI somehow?”

Well, first of all, it’s not nice to be sarcastic. But you’re not wrong. I can’t help but see, in GeoGuessr, the quintessential example of the world as data. The world made digital, dissolved into a vast ocean of numbers, so that we can train on it, discover the hidden patterns that give it structure, and reassemble it into a new kind of space. From one direction we struggle to build synthetic minds that contain a reliable world model. Simultaneously, from the other direction, we want to feel what its like to model the world like a learning machine in an adversarial network, to fine-tune our minds to its latent frequencies, to become cosmopolitan globetrotters without leaving our computers.

GeoGuessr was invented in 2013 by a Swedish coder named Anton Wallén, and over the past decade has steadily grown from an amusing novelty into a serious game with a dedicated community of players, some of whom (like the supremely likable Tom Davies aka GeoWizard) have built tiny media empires around the game. When he’s not playing GeoGuessr, Tom invents crazy, real-world challenges for himself, like attempting to trek across countries in a straight line. He also attempts to pinpoint the location of photos sent in by viewers, scouring them for clues and then using all the resources of the internet to reverse-engineer their position.

Inventing ridiculous problems is what games do best. GeoGuessr is an example of games growing like moss on the newly exposed surfaces created by the internet as it carves up the world into information. How good are you at seeing and navigating the world through these new tools, at wayfinding in this hybrid realm? How good are you at search? At database orienteering? At prompt engineering? These skills are too important to teach in school, but games can’t help themselves, and rush in to fill the void. See, for example, this guy, who hunts down sports broadcasts glimpsed in the background of movies and TV shows:

@noproblemgamblerReplying to @kurtisflo This is the most blind research i have ever done in a video #baseball #ncaa

Tiktok failed to load.

Enable 3rd party cookies or use another browser

GeoGuessr is the crowning jewel of this emerging genre, and I’m happy to see this free browser game, this great-grandchild of Where In the World is Carmen Sandiego, given the full on, high-pressure eSports razzle dazzle. The hype is real.

Picture yourself sitting in a darkened movie theater, as the first trailer begins to play. It’s a historical drama about naval warfare during the Napoleonic era. What do you see?

What kind of question is that? It’s your hypothetical! I see some boats, the ocean, movie stars, you know, typical movie trailer stuff.

Yeah, that’s how it works for me, too. But now, imagine that you’re a historian who is an expert on the tactics of 19th century ship combat. Imagine how your eyes would light up when this trailer started, imagine what details you would notice.

Now imagine that you are a special effects programmer who specializes in fluid dynamics and water simulation.

Now imagine that you are a costume designer, or a vocal coach, or an optical engineer who designs movie projectors, or someone who edits film trailers for a living.

In each case, you would see the same thing, in terms of raw sensory input. But what you saw would be totally different, in terms of the information you were receiving. You would see things that are invisible to me, because I don’t notice them, I don’t have the context that would make them salient to me.

Where I just saw some boats, you would see barques and barkentines, schooners and brigs. You would see the tell-tale artifacts that result from using volumetric grids to approximate the Navier-Stokes equations. You would see cuff widths and ruffles, collar heights and lapel angles. You would notice glottal stops and plosive t’s, lumens and resolution and color depth and contrast ratio, establishing shots and jump cuts right where you expect them or, surprisingly, not.

In each case, the trailer would be part of a different conversation, raising and answering an entirely different set of questions, above and beyond the standard ones of “who’s in this movie?”, “what happens in the story?”, and “do I want to see it?”

Now do the same exercise but imagine being in a grocery store or a hospital waiting room, imagine being an ad executive, a homeless person, an architect, an actor, a child, a visitor from a distant country or planet, an artist. Imagine each perspective snapping into place like the lenses of an optometrist’s machine. Each snap causing certain objects and details to jump forward into sharp focus while others disappear.

In psychology, the term affect refers to someone’s overall feeling, emotional state, disposition, or mood. In philosophy, it is used more broadly to refer to the connections and attachments we have, or could have, to objects in our environment and the way those relationships are expressed in our bodily experience.

But psychology and philosophy are just subdomains of computer science now. So I propose we examine affect as a type of protocol, a method of formatting and processing data. Your affect, your attitude, your stance in relation to the world, determines the signals you receive from your environment, and shapes the information you are able to extract from it.

We can see the world as data, but we must remember that it doesn’t come that way. Data is constructed, it is manufactured by instruments and conventions, and, for humans anyway, affect is a central component of how the world becomes data to us, how the world communicates with us, how we perceive it, make models of it, learn from it.

Affect, in this regard, includes your mood, your emotional frequency, but it also includes your projects, your commitments, your interests, habits, hobbies, profession, relationships. These things combine to create a dynamic, multi-layered sensory apparatus that organizes the infinitely ambiguous stuff of the world into an ordered array of objects, patterns, and ideas that can be noticed, observed, and understood.

Like all protocols, the protocol defined by your affect is a trade-off. It allows for the transmission of certain objects, patterns, and ideas, and filters out others. Unlike HTTP or SMTP, your affect isn’t compact, fixed, and precise, it is messy, organic, evolving. It wasn’t designed, but it isn’t entirely beyond our control. The fact that we can imagine it taking on different forms shows that we have some influence over it. And how would we change it if we could?

I’ll tell you what I want. I want it all. I’m greedy. I want to see everything. I know this is impossible. I know that to see anything at all means to see some things and not others. Still, the heart wants what it wants. A loss minimization function can dream can’t it?

One of the challenges of hacking your own affect protocol is that its primary dimension is valence — the dimension of good/bad, love/hate, approach/avoid. It’s obvious why this would be true for the sensory functions of any living creature, evolved or designed — to exist at all is basically to prefer some arrangements of the world over others. “You can’t get an ought from an is” they say. What they don’t tell you is that you can’t ever pry the two apart either.

Maybe the ability to see requires wanting to see some things and not others. Maybe we can only perceive the world when it is splayed out against a graph of fear and desire. Not just because we evolved to see, and avoid, tigers (although we very much did) but because, on a much more fundamental level, the whole idea of a signal starts with a particular arrangement of chemical gradients in a cell’s environment and the vibration of cilia orienting and propelling the cell, towards or away from. Affect begins here. We didn’t start by modeling the world and then discover there was a sandwich in it. We started with the sandwich and the world came along for the ride.

But now that we have the world, do we still need to hang, quite so tightly, onto fear and desire, disgust and pleasure? What if we just had desire and pleasure? Wouldn’t that be nice? Or would that break the spell?

Pick up a large rock and turn it over. The underside is a writhing mass of wriggling insects. Yecch. Now dial up the affect of an entomologist, like Trinity, in the Matrix, downloading the helicopter pilot app. There’s no danger here, no discomfort or disgust, just fascination, enthusiasm, and the potential of discovery.

Why do we find boring business meetings so painful, and a lonely pond at sunset so pleasing? Is the croaking of frogs really that much more interesting than the chirp and drone of middle managers discussing quarterly reports? Why can’t we see this, too, as nature sorting itself into delicate, complex, semi-stable equilibria?

Is it that we are in danger of forgetting that we shouldn’t eat contaminated food? That boring business meetings are bad? Really? Do you think that, if I gave you the superpower of appreciating the pond beauty of business meetings, you would spend all your time in them? Of course not! You would be too excited to try out this superpower on museums and libraries, on kindergarten concerts and family reunions and busy street corners and starry skies and lecture halls and crowded bazaars and lonely ponds!

If I could guarantee you that the knowledge that bug-repulsion and meeting-hatred is trying to teach you is safe, that you could retain all the benefits of fully understanding the problems they point to, while, at the same time, seeing them more clearly, understanding them more deeply, wouldn’t you take that deal? Won’t you learn better once you start learning because you love to learn, not just to avoid punishment?

Here’s another kind of valence trade-off: do I have to hate dumb stuff? If I learn to appreciate fine wine, say, does the cheap stuff, which I used to enjoy, have to taste bad to me now? If so, that seems like a bad deal. I get that, given the opportunity, once you have the capacity to appreciate the good stuff — the more complex, subtle, refined, whatever — you should steer towards that. But a lot of the time it’s not your choice and you find yourself in a situation where drinking the cheap stuff is your best current option. Does it have to taste bad to you now? Wouldn’t your life be strictly better if it didn’t? If you could have all the benefits of enjoying the good stuff at the highest level, the ability to distinguish all the subtle differences, and appreciate exactly how they work together to create a sublime experience, and still love the cheap stuff? Really love the taste of it?

Also, isn’t that the kind of person you want to be? Like Nick Charles in the Thin Man movies, moving effortlessly between the rich snobs of high society and the crude slobs of the underworld and never not having a good time, never not enjoying the company.

I guess it’s possible that without the spur of distaste driving us away from the cheap stuff we would never bother. But isn’t the lure of the good stuff enough, on its own? Isn’t it enough to know why all our friends hate Skrillex? Do we also have to be the sad guy standing in the corner at the party who refuses to dance?

As someone who plays poker, I can’t really enjoy the poker scene from Casino Royale. It’s meant to show James Bond’s skill at the psychological metagame of poker, but really he just gets dealt the nuts when his opponent has a very strong hand. Literally anyone would get all the chips in this spot. My poker knowledge allows me to see and understand aspects of this scene that would otherwise be opaque to me but it also just kind of spoils it.

I think my inability to enjoy this scene is a mistake, it dulls my ability to appreciate the lighting, the set, Daniel Craig’s performance, the fluid cinematography of Phil Méheux, the complex choreography of explanation that makes the game legible to non-players, the way replacing Baccarat (from the book) with Hold’em subtly changes the meaning of James Bond as a character. I want to be able to see this scene as a fan who loves it, who knows how to love it, and also be able to see it as a sophisticated film connoisseur for whom it is, perhaps, just another example of formulaic, big-budget, Hollywood filmmaking (but what a nice example!) I want as many of these perspectives as I am capable of processing available to me, because they are all ways of getting information out of the scene.

Ironically, if I were trying to create an AI system that could see, and reason about, the world more effectively (and who says I’m not!) I would be trying to find ways to inject more of this kind of embodied, valence-laden affect into its perceptual apparatus. But thinking about my own experience in light of machine learning, personally, I want less.

Protocols are never private affairs, of course. They are, by definition, shared agreements. Consider this my proposal, not for sweeping changes, but for an incremental adjustment that I believe could yield dramatic improvements. Mostly, consider it yet another re-statement of the fundamental message of this blog: aesthetics matter.