What picture of the world do Newton’s laws evoke? Here I’m not interested in the specific equations of motion he wrote down, or his technical definitions of force, acceleration, and inertia. I am interested in the intuitions and metaphors underlying his theories. As humans, we understand everything by metaphor1 to our basic, everyday experiences, and for most people, Newton’s laws draw on very particular metaphors:

The world is like a bunch of billiard balls. What are the salient properties of billiard balls from our perspective as humans? First, the balls are atomic: they are indivisible and featureless units. There is nothing too interesting about them beyond their interactions with other balls. Also, they are inert and unalive. They don’t feel anything of course, they’re just rocks.

And what about their movements and interactions? Their movements and interactions are all predetermined. Strike the balls in the same way, and they will follow the exact same paths. Also, their interactions are all local: the only thing influencing the movement of one ball is the direct contact it has with other balls (or with the edges of the table). Everything that happens on a billiard table is just a bunch of bumping and shoving. Out of that bumping and shoving comes everything we care about in a game of billiards, all the patterns and strategies and gameplay narratives. It’s a bunch of inert, uninteresting, featureless rocks bumping and shoving against each other.

The “scientifically minded” person who has not spent much time thinking through their metaphorical handles on reality sees all of these things as properties of the world in general. The world is a bunch of billiard balls that move. The world is a bunch of puzzle pieces or lego blocks. The world is fundamentally dead, predetermined, and unwittingly, imperturbably subject to the progression of math equations. What’s interesting is that even after realizing that the theory itself is technically untrue (quantum mechanics has superseded Newton’s laws), the metaphors underlying the theory stay with us.

There is a philosophical name for the worldview that results from this picture: mechanism. And mechanism, as a way of looking at the world, is one of the most powerful things we have ever come up with. Thinking of the world in terms of local bumping and shoving, featureless subunits that grind against each other like gears and pulleys, has given us, well, incredibly useful inventions. And it continues to be a useful orientation for science today: we seek a mechanistic understanding of black box AI systems, and we try to decode the marvels of life through mapping out its biomechanical foundations – the shapes of proteins, the graph of neurons in your head.

Of course, we also know today that mechanism is not the full picture of reality. The elementary particles of reality are not actually featureless (and they might not be unfeeling), local “bumping and shoving” is not the only legitimate form of interaction, and the universe might not actually be fully determined by its initial conditions. Research programs that operate exclusively in terms of mechanism have often run into dead ends, like with the recent slowdowns in progress in mechanistic interpretability, or the failures of reductionism in biology.

But there is something else that makes the mechanistic picture especially hard to let go of: the possibility of control. That is the promise that mechanism makes: once you truly have a mechanistic understanding of something, you can interfere with it, disrupt it, reshape it to fit your goals. This is why AI safety researchers care to have a mechanistic understanding of AI systems, so that we can maintain control over them, lest they become too powerful.

To let go of mechanism, then, is to accept that there is a limit to your capacity to control the world. The mechanist believes that reality is a mechanism all the way down, which means that reality can be controlled all the way down. The more we progress in science and technology, the more fully we can control reality to fit exactly to our liking.

I don’t think we can entirely settle this question today, with philosophy – history is replete with both examples of our failure to shape reality to our liking (see every dictator that was toppled, the long list of diseases we have yet to cure, and the fact that we still can’t get computers to work right), but it is also full of feats of control we would not have imagined possible (gene editing, moon landings, and short films made out of individual atoms). We may just have to let the mechanists have their chance at building more technology and see how powerful it makes us. But there is another question I want to leave you with, which might be easier to answer today. What kind of person do you become, when mechanism and control are your operative lens for viewing the world? Is that the kind of person you want to be? And is that the kind of person you want to share a world with?

It’s commonplace now to think of living things as machines. A plant is a machine that converts sunlight, water, and carbon dioxide into oxygen via photosynthesis; an animal is a machine that takes oxygen and food and outputs carbon dioxide and water. While there are parallels between machines and living beings, most people, especially people who primarily work with machines (i.e. engineers), take this analogy too far. There is a crucial difference between the two that is not captured when thinking about inputs and outputs. In living things, unlike in machines, unreliable parts make a reliable whole.

All of us are made of building blocks which have an autonomy of their own, but which nonetheless work together to create an integrated system that functions consistently. Cells, proteins, chemical cascades, and neurons all have an element of chaos to them—they don’t behave in easily predictable ways—and yet your heart beats three billion times, continuously, until you die.

We are unlike machines in that our parts are not reliable: if you give the same neuron the same stimulus, it will not always respond in the same way. We are also unlike machines in that our parts don’t have a perfect fit: people often talk about proteins binding to each other like a “lock and key”, but it’s more like a handshake, where two proteins modify each other’s shape as they interact, connecting based on a level of “binding affinity” that can be higher or lower but never perfect.

It’s hard to overstate just how different this is from an integrated circuit, whose parts are precision-manufactured to fit, and are built for consistency. If you set a bit in a CPU register to a 0 or 1, it will maintain that state with 99.9999% reliability: in fact, the rare time this fails is in the case of cosmic rays causing bit flips. It takes interference from other galaxies to throw the building blocks of computers out of sync, and even that only happens about once every quintillion operations.

The unreliability of biological building blocks is not a bug, it’s a feature. Cells are better thought of as active agents which pursue goals than passive mechanical parts. This means that on the one hand, they are less reliable, because they may sometimes take actions that are misaligned with the goals of the larger system (e.g. cancer). On the other hand, this unreliability comes with a creative flexibility: when one part of the system makes a mistake, another part of the system can step in and compensate. When neurons in your visual cortex are damaged, neighboring neurons can reorganize to take over their function, restoring partial vision. As cells in every part of your body die, other cells come in to take their place. Fault-tolerance is built into every level of the system, because every level of the system is partly unreliable. The unreliability of the lower levels, rather than being a nuisance, can actually be harnessed, like when bacteria increase the amount of randomness in their gene expression when stressed, to increase their odds of survival.

This is the root of the difference between biology and machines, at least in their current form: biology is grown, machines are designed.1 Machines are built by teams of people, and as such they need to be shaped in such a way as to be easily understood and manipulated by teams of people. The principles that apply in an engineering context—separation of concerns, reducing the system down to simple building blocks, each part having a well-defined function, each part being made to be as reliable as possible—apply only partially in biology, forever restrained by the lack of any conscious thinking minds behind the development of life. The world of biology is not a world of easily distinguishable objects and well-defined relationships, the kind you’d find in an architecture diagram. The world of biology is one of interleaving, self-perpetuating, ever-evolving processes.

Ultimately, this difference between organisms and machines, like everything else, is more a matter of degree than a strict binary. There are evolutionary aspects to the development of technology; a software codebase consisting of millions of lines of code is not designed top-down, but emerges bottom-up as thousands of individual engineers contribute to it. The weights in a neural network that powers a chatbot are not “designed”, they are grown as part of a training process, bringing them closer to biology than traditional software.

But there is a useful difference to keep in mind here, which is less the difference between groups of objects (cells vs computers, plants vs cranes), but between mindsets of understanding and design. There is the mechanical mindset – which emphasizes reduction into simple parts and total control over every aspect of the system – and the organismal mindset – which emphasizes creative autonomy over predictability. What we need to be careful about, especially in a society overrun by machines, is letting the mechanical mindset dominate the other. To view predictability and control as the unalloyed good, and to think of randomness as a problem to be eradicated. To forget that resilience emerges not from rigidity but from flexible adaptation. It’s easy to look at our computers and skyscrapers and admire them for their elegance, contrasting them with the messiness of a neural wiring diagram or a biochemical pathway. It’s harder to see the wisdom in nature’s messiness.

Thanks James for feedback on drafts.

Epistemic status: playing around with ideas here, let me know what you think.

Here’s something I have more clarity on now – good thinking is not just about being rigorous. There was a very long while where I thought of rigorous thinking – the kind involved in writing math proofs – as the most elevated kind of thinking. I believed, for example, that a reliable way to become a more “brilliant” person, to refine your intellectual capacity, would be to do more and more hard math. Why did I think this? There were two misconceptions at the root of it.

One misconception was the idea that math itself is a purer or more elevated kind of truth. Of course, math is indeed pure; “correctness” in math is clearer than all other disciplines. But that is because it is devoid of meaning!1 You can call math the study of necessary truth; equivalently you could call math the study of the mechanics of meaningless symbols that follow axiomatic rules. Math is a powerful tool but it's really only one way of trying to grapple with the world.2

The other misconception was just that because math is so hard – it’s probably uncontroversial to say that pure math and physics are the hardest intellectual disciplines you could try to master – that this hardness also elevated it in some way. The fact that a field is “more challenging to understand” than other fields does say something, of course. But what exactly does it say? Does the fact that it’s more challenging lead you to actually get better at all other fields, at other kinds of thinking?

What does it mean to be intellectually capable? There are many different ways of putting this. How many novel ideas have you put out there that have changed people’s thinking? How many discoveries have you made at the frontier of human knowledge? How many great essays/lectures have you produced that were viewed as insightful by other people who have produced insightful essays/lectures? How good are you at asking questions? How good at you at identifying ideas that are likely to be consequential in the future? How good are you at spotting intellectual grift?

If we loosely bucket all of these skills as one vague concept of “thinking clearly,”3 what I want to say is that simply doing more math does not lead to clearer thinking. How do we make this case? One empirical observation is that mathematicians are not regarded as being uniquely good at everything else. It’s not true that you see PhDs in math going on to establish a disproportionate number of groundbreaking discoveries in all other fields.4

That’s an empirical argument, now here’s a more psychological point: the fact that math is fundamentally about following logical rules, means you don’t get to do as much “flexing your intuition muscles”; thinking in math is a way of thinking very rigidly.5

Creativity and insight require the opposite of mental rigidity – they require mental relaxation. Insight is fundamentally a kind of analogy-making – forming previously unseen connections between disparate ideas.6 How do you make connections between disparate ideas? There is some kind of role here for relaxation. I’m not exactly sure why, but relaxing seems to allow for unexpected connections to be made. Consider the fact that our best ideas often come to us when we’re not thinking hard about them; consider Ramanujan's dreams, or the role of LSD in Francis Crick’s discovery of the DNA helix or Karry Mullis’s invention of the polymerase chain reaction (PCR), and the vast literature on how psychedelics increase plasticity in the brain.

There are also arguments that mental relaxation is crucial for psychological change—so mental flexibility is important not just for intellectual insights but also for emotional ones.

I think to foster more insight you need to dance. Math (or, precise and rigorous thinking more broadly) is like a weight-lifting of intellectual activity – you are building up pure muscle mass. But intellectual insight also requires dexterity – which you get more from dance and stretching. You want to marinate in your intuitions, appreciate art, relax, dream, doodle, journal, roll around on the floor, listen to music, and (quite literally) dance and stretch.

There’s an ironic historical note here, pointed out by Julian Gough, which is that math was originally meant to be a “humble language,” delimiting science to a narrow domain (that which can be described by math) and leaving all the other “important stuff” in the hands of the Catholic Church. This view has now been turned upside down: many people think of math as the only language of truth, or as the purest form of truth. Gough calls science’s insistence on mathematical language as a “vow of poverty.” At least for me, it was revelatory that I had never even contemplated this view – that math might be a deeply limited language rather than some kind of superior language. Math is a language of quantities devoid of qualities. It’s a language of form with no substance. It’s syntax without semantics.

The question remaining here is: what role should math play, in a serious intellectual engagement with life? I do think that having done many proof-based math courses (and also having been a programmer for over a decade), I have learned useful things. Continuing the acrobatics analogy, I've built the necessary muscle mass for certain things that will just be hard to have any intuition about if you've never done hard math or programming. Examples: loops, recursion, self-reference/strange loops, proof-by-contradiction, uncountable numbers, diagonalization, dimensionality, orthogonality, continuity and differentiation, the idea of “invariance under stretching and bending” (topology), and “invariance under rotation and translation” (geometry), infinitesimals. Debatably, you could get a feel for all these things without ever having done the actual math or programming; I'm not sure. But having done it certainly helps.

I think a reasonable approach to math is to learn it to the extent that you need to, for the questions you want to ask, and not feel any obligation to go further than that. For example, I am somewhat convinced that understanding the physics of fields could potentially be relevant for consciousness (at least, the QRI folks claim that consciousness could be explained by electromagnetic field topologies) and so that's something I’m interested in. They’ve also piqued my interest in group theory and symmetries, since they think of valence of conscious experiences as having to do with symmetry.

There is an unanswered meta-question here, which is “how far will formal methods take us?” The big shift for me the past five years has been losing my unquestioned faith in formal methods as “the correct way to think about ultimate truth.”7 I am less convinced that the way to understand the world is to make formal mathematical models of it and prove theorems about those models. The way I could be wrong is if further study of pure math led to some groundbreaking discovery that had tangible implications for our understanding of the world. Some people believe, for example, the universe is itself a mathematical object, and in that case, there is a very strong case for doing as much math as possible because you are studying the fundamental structure of the universe itself.

My friend Will recently made the following observation about AI models: “it's interesting that 1.5 billion parameters is all you need to crush math competitions, but you need like 15 trillion to make the model be funny. maybe humor is the right measure of true intelligence.” I think it was said in jest, but I wonder if there’s some deeper insight there. An insight that sounds like something like: that which is formalizable is not necessarily that which is most powerful, or most important to know.

Thanks to Santi for discussion on a draft.

I’d like to describe how my attitude towards reading has shifted in the past few years, in a direction that has made me a more effective thinker and writer. It’s a shift from reading as passive entertainment to reading as creative activity. This is specifically in the context of “intellectual work”—the kind of work I try to do in this newsletter, which is to explore ideas, understand them deeply, and explain them well. In the past year I’ve had notable success with “contributing to the online conversation,” through e.g. this post that made it to the front page of hacker news and this post that sparked lots of animated conversations on twitter. I credit a large part of this success to my shifting habits around reading.

The notes below are largely written in second-person “you should do X or Y” language, but none of the advice is meant to be taken too seriously; take this as a list of invitations to consider and see how well they work for you.

Reading is fundamentally not about the book; it’s not about going through the contents of the book in linear order and absorbing them. The thing that matters most in reading is your response to the text you’re engaging with and the questions you’re trying to answer. That’s how you should be thinking about reading – rather than being primarily occupied with what the author is trying to say, shift your focus to your own questions and response to what the author is saying. Treat reading like a conversation. What do you find clear and what do you find confusing? Is what the author’s saying right or wrong? How does this tie in to the other ideas and questions you’ve been thinking about?

Focus your efforts not on “finishing books” but on “answering questions.” Keep the questions you’re trying to answer top of mind. Every now and then, make a list of all the things you’re most curious about, and as you ponder the questions you can list the books or papers that seem relevant for each. The questions are primary; the books and papers are secondary. (Once you’ve formulated your list of questions and reading trails, you don’t necessarily have to take any action on it. It just helps to bring all the subconscious stuff to the surface, so that it can more effectively guide your intuitive choices around what to read next.)

Your reading list will be effectively infinite. Don’t attempt to keep this list organized. I used to maintain an organized database of more than a five hundred book recommendations in Notion, categorized by topic and author etc, and I barely touch it these days. Nowadays whenever there’s a book or paper I’d like to read I just make a note about it in my daily notes. But importantly, I don’t just jot down the name of the book/article – I also jot down why I’m interested in reading it in that moment. What question do I imagine it will answer? How do I expect the book to change me? I find that clearly stating the feelings and reasons behind my interest helps me prioritize more effectively, and it helps me relax about the high likelihood that I will never read the thing I’ve just jotted down.

Notice and relinquish your psychological attachment to the “book” as a form. The book is not the point of intellectual work (neither is the paper); it’s merely a tool that can be used in the service of it. It’s been incredibly helpful for me to slowly purge my romantic preciousness around the book as a physical object. In the past I’d keep all my books in pristine condition, never highlighting or annotating them (even for my PDF files!). Nowadays I highlight and annotate like a madman, though I’m still a little too obsessive about keeping my physical books tidy.

Don’t get too attached to your notes either. I’ve expended lots of effort on making my notes pretty, organized, and extremely thorough (see e.g. my notes on Beginning of Infinity or Bear’s neuroscience textbook), and this hasn’t proved particularly helpful to anyone. Aesthetics matter, but only for the things you or others are likely to look at often; most of your notes won’t meet this condition.

More thoughts on note-taking. In the past, when I took notes I was primarily focused on faithfully restating what the author said. This involved summarizing and saving lots of long passages from the book. Restating what the author said in your own words is good, but saving long passages is generally not – again, you likely won’t use those passages for anything in the future. (Unless you found a passage so good to the point of getting emotional about it, or you think it serves as a useful example of “good writing” to analyze for its technique.) In addition to summarizing, it’s very important when taking notes to editorialize – what do you actually think (and feel) about what they said? Was it thrilling or was it annoying, and why? Try to be specific – rather than “that was interesting,” try “I was interested in his depiction of Scholasticism and how wildly different it is from our current worldview.”

I’ve made this mental list of “what matters when reading,” which helps me avoid spending too much time on the wrong thing. In order, what matters most to least is:

• the output of your reading process (i.e. essays)

• the existence of the ideas in your brain

• the notes you take in your notebook

• the list of books you’ve finished

This is another instance of making the consumption less central and the act of creation more central. Your notes on the book are more important than the fact that you finished the book; but also, the way the ideas integrate into your mind (and actions) is more important than any stale notes you take; but further, it’s important that the ideas affect not just your mind but also other people’s minds. This is an act of service! You are trying to be a channel for good ideas in the competition against mediocre and bad ideas.

Now, one might complain that this way of thinking is very utilitarian: does reading only matter to the extent that it helps you write essays and change other people’s minds? Yes and no; I still read purely for leisure (currently enjoying American Pageant, A Tale for the Time Being, Alberts’ Essential Cell Biology, Hofstadter’s Surfaces and Essences). But also, I care about writing good essays, and I care making our culture better, and that requires a slightly different mindset than pure leisure when it comes to reading.

Let me give an example: before I started writing the Michael Levin essay, I set an explicit intent to understand what Michael Levin’s work is about and why it’s important, with a background goal of potentially writing an essay about it if I find something interesting. Now, this is a very different goal from “I just want to read Michael Levin’s papers for fun.” If the goal is fun and leisure, then I like going through papers linearly and just enjoying myself. But when the goal is answering a specific question, I take a much more active approach (in line with everything I’ve described): I’ll jump around between materials more, I’ll constantly be making note of the “questions I currently have and what to read next to answer them,” and I’m thinking often about “what are the most interesting tidbits I might want to include in a potential essay.” The point is, it’s a much more effortful process. If I just wanted to have fun I could spend an entire year just meticulously reading all of Michael Levin’s papers, and that could be nice,1 but doing so would be antithetical to the objective I had. The point is to be clear with yourself about when you’re doing “work” and when you’re doing “leisure,” and cleanly separating the two.

Another important shift when reading is to not get “bogged down in prerequisites.” This is the impulse to make sure you’ve “caught up on all the literature” and “gotten all the required background knowledge” before you even contemplate asking questions of your own or contributing your own ideas. It’s understandable to think this way – there is so much that has been written, clearly every question we want to ask must have already been answered somewhere, right? Except no – there are many answers out there that the experts have not figured out yet, and that even you – yes, you, an amateur – can figure out. You can give yourself permission to do your own research, and just assume that you might be able to move to the “frontier of our knowledge” without a decade of preparation. Maybe you will be wrong, in which case you’ll learn something new; but if you are right, you will have discovered something groundbreaking.

Being more engaged and creative as a reader is a one specific instance of having more agency in general. It’s taking the driver’s seat with regard to your intellectual growth rather than merely letting someone else dictate it. My hope is that you learn this lesson earlier than I have. There is so much to discover, there are still ideas out there that are easy to find. Go forth and explore.

Subscribe now

I. Introduction

For the reader: this is about some of the technical details of David Deutsch's philosophy. If all you took away from Deutsch is that optimism is good, problems are solvable, and humans are significant—those things are kept in tact, and you can carry on with your day. But, if what you took away from reading Deutsch was an entire philosophy of knowledge, a worldview that percolates deep into your understanding of humans, truth, and reality—then I have some bad news for you.

I should include some background on myself: I originally discovered Deutsch about four years ago, and I became obsessed. I read multiple of his and Karl Popper’s books, papers, lectures and interviews, and took many thousands of words of notes on all of it. I really felt like I was coming face to face with a profoundly significant set of ideas—a deeply clear, comprehensive, and coherent picture of reality. I now believe this was wrong, and that I had fallen prey to a number of fairly obvious conceptual and psychological errors.

A note on terminology: in this essay I’ll be grouping together the ideas espoused by David Deutsch, Brett Hall, Karl Popper, and Chiara Marletto into one loose bucket, which I’ll refer to interchangeably as “critical rationalism,” or the “Deutschian” or “Popperian” worldview. While I recognize they don’t all have the exact same position on every question, they share enough common philosophical ground to be grouped together for the sake of argument.

Put most succinctly, the critical rationalists’ mistake is that they view the world in terms of simplistic dichotomies. They formulate theories for how things work—say, how humans obtain knowledge about the world—and then fit every conceivable piece of evidence into that theory, making their ideas effectively immune to criticism. When taken seriously, their ideas lead them to a very distorted understanding of how humans work, and how we know things about the world.

The way I’ll make many of the arguments in this essay is oblique rather than direct. Some of my points will be about the meta of Deutsch’s statements rather than the substance of them; I’ll even make some sociological observations rather than purely technical/philosophical ones. I think this is necessary because, as I’ll try to demonstrate, critical rationalism is based on a set of mutually reinforcing assumptions and framings that are effectively impossible to refute all at once, purely on the substance of their claims. There is no single, definitive contradiction within the philosophy—if there was, Popper and Deutsch themselves would have found it long ago. But it’s possible to step just outside of the philosophy—to see the web of framings, rationalizations, and everyday behaviors that make up the worldview—to see the problems in it. The Deutschian worldview is a kind of philosophical “local maximum,” from which it’s impossible to move towards greater conceptual clarity without first taking apart a lot of conceptual scaffolding.1

A big part of my motivation for writing this essay is psychological: what I would like to do is help a version of myself three years in the past who, by virtue of being so absorbed in the Deutschian worldview, felt a sense of existential loneliness, and got into a number of silly arguments in which he tried to convince others of this worldview (you might go as far as calling it a religion), and failed, mostly because in all these conversations he and his counterpart were talking past each other. I'm trying to help a past version of myself waste less of his time and emotional energy on silly arguments trying to defend Deutsch.

What I am trying to do is also sociological—I'm trying to explain the apparent contrast between a large number of people who are convinced that Deutsch's books (The Beginning of Infinity in particular) are some of the most powerful and important books ever written, and an even larger part of the population (including much of academic philosophy and a decent chunk of the scientific community) who tends to respond to Deutsch’s work with ambivalence, appreciating its value but not recognizing it as revolutionary. I am trying to explain why it is that some people come away from reading Deutsch totally transfixed and transformed, and others come away feeling somewhere on the spectrum between “this was silly” and “it was pretty good I guess.” It is not, as many Deutschians believe, simply a matter of the latter group not understanding what Deutsch is saying. There is something deeper going on.

Now, Deutsch and the critical rationalists deserve a ton of credit: it’s a fundamentally well-intentioned way of thinking. There are many basic tenets of theirs that I still agree with: knowledge is possible and it’s good; humans have an important role to play in the universe; the scope of what we can know about the world is unimaginably large; technological progress is a good thing and much more of it is attainable. The thing is, all of these principles remain true, even without all the tortured philosophizing that Popper and Deutsch do. Critical rationalism adds an unnecessary, pseudo-rigorous veneer onto these more obvious and basic principles.2

By the nature of Deutsch and Popper’s ideas being abstract, this essay will also necessarily be abstract. To combat this, let me ground the whole essay in a concrete empirical bet: Popper’s ideas about epistemology, and David Deutsch’s extensions of them, will forever remain in the footnotes of the history of philosophy. Popper’s falsificationism, which was the main idea that he’s widely known for today, will continue to remain the only thing that he’s widely known for. The frustrating fact that Wittgenstein is widely regarded as a more influential philosopher than Popper will continue to remain true. Critical rationalism will never be widely recognized as the “one correct epistemology,” as the actual explanation (or even the precursor to an explanation) of knowledge, progress, and creativity. Instead it will be viewed, like many philosophical schools before it, as a useful and ambitious project that ultimately failed. In other words, critical rationalism is a kind of philosophical deadend: the Deutschian deadend.

II. Fitting everything into your theory

Critical rationalists formulate theories for how they think certain things should work, and then they view everything as an example of that theory.

We can start with one of the most basic of Popper’s ideas—that of conjecture and refutation—which Chiara Marletto describes in her book The Science of Can and Can’t:

Given that knowledge has such an essential role in the survival of complex entities, it is essential to understand the process by which new knowledge is created from scratch in our mind. Fortunately, this process was elucidated by the philosopher Karl Popper in the mid-twentieth century. He argued that knowledge creation always starts with a problem, which we can think of as a clash between different ideas someone has about reality. For example, when writing a story, the clash in the author’s mind might be between the desire to use elegant, lyrical language and the necessity of keeping the attention of the reader alive with a gripping plot. The author has to find a way of meeting both these criteria, which may clash in certain situations: a long passage describing an idyllic landscape might give a perfect chance to meet the former criterion but might result in the reader dropping the book and switching on the TV, because it slows down the pace of storytelling. To address problems such as this, one has to create new knowledge.

Marletto claims that Popper has “elucidated” the process by which new knowledge is created. But has he really elucidated anything? This passage exemplifies one of the central critical rationalist tendencies: framing your theories in terms of highly abstract concepts, which then allows you to frame any conceivable situation as an example fitting your theory. “A clash between different ideas someone has about reality” can be applied to any conceivable human experience. You could describe anything as a conflict between two ideas. Me wanting to write a book is a conflict between the fact that I want to write a book and the fact that I haven’t written a book yet. This is not to say that it’s untrue: there are of course conflicts in our mind all the time. But this alone doesn’t bring us any closer to an understanding of how we create knowledge. It’s like saying: “humans learn about the world by asking questions and then answering them.” Great, now what do we do about this?

What makes critical rationalism appealing is the way it seems to “unify” disparate ideas. It literally unifies all of human activity into one big category of “knowledge-creation via conjecture and refutation.” As you’re going about your day and walking around, you’re creating conjectures and criticizing them; when you’re working through an emotional problem, you’re creating and criticizing conjectures; if you’re talking to someone, you’re creating and criticizing conjectures; if you’re dancing or making art, you’re creating and criticizing conjectures. Evolution is also doing the same thing—creating and criticizing (implicit) conjectures. Everything is about knowledge: DNA contains knowledge, technology contains knowledge, all of culture is a bunch of knowledge. What distinguishes good art from bad art is the fact that it has “more knowledge.”

If you’re into neat conceptual unifications and have a fetish for “grand unified theories that explain everything” this will be very satisfying—but again, it’s not clear what we actually get from this grand unification. The question for critical rationalists is this: what specific advances have we made across fields as diverse as fiction-writing, painting, biology, neuroscience, and music composition, by virtue of viewing all of them as instances of “conjectures and refutations”? I argue that because the model is at such a high level of abstraction, it hasn’t actually been useful across all the domains it claims to apply to.3

To be fair, Popper and Deutsch’s epistemology goes deeper than this one idea. To see how this “rationalizing reflex” is present throughout Popperian thinking, it helps to look at some of the other core principles that critical rationalists espouse.

Observation is not always theory-laden

There are two ideas are the core of Popper’s epistemology which are, as far as they go, true, useful, and eloquently put—except that critical rationalists, once again, over-use and over-apply them.

The first is that “observation is theory-laden.” Here Popper is saying that there is no such thing as a “totally pure observation statement.” A statement that you might naively think of as purely observational—“the temperature of the water in this cup is 40ºC”—is actually loaded up with a bunch of implicit theories and concepts, like the concept of water, temperature, dimensions, the notion of “celsius,” the idea of numbers, and the idea of a “cup.” So when you make such a statement, it’s not a pure objective observation because it carries this baggage of implicit theories with it. There are even theories embedded in the very structure of the language you’re using to make the observation—e.g. the subjective-verb-object structure of clauses in English.

Why is the theory-ladenness of observation so important? This brings us to the second Popperian principle, which is that “the truth is not manifest.” Popper stakes this claim in contrast to some other philosophers who claim that we have “direct access” to the truth of the world through our senses. Popper very rightly points out that the truth is not obvious, because we can be deceived in many ways. There are countless examples of this, from mundane optical illusions, to errors in our measuring instruments, to superstitions that taint our experience of the world. All of these are “sources of fallibility”—they act as barriers to our “direct access” to the truth.

Now, to see the limits of both of these claims—that observation is theory-laden and the truth is not manifest—you need to appreciate the underlying frame in which they’re made. The underlying assumption here is that we, as humans, are irrevocably “cut off” from the actual objective world, only making tentative conjectures about it, unable to ever “verify” anything we know. Critical rationalists literally claim that we can never assert the truth of anything—not even the most basic observations like “it’s Wednesday” or “I’m alive.” For critical rationalists, not only are we prohibited from saying these statements are true—we can’t even say that they’re likely to be true. The entirety of our knowledge is, as they put it, a bunch of “unjustified untruths.”

The critical rationalist’s picture of the world is one of radical skepticism, born of a metaphysical tradition dating back hundreds of years to Descartes and others, which Deutsch and Popper never seriously question. In this picture, we are always “groping around in the darkness of a cave,” mired in our “infinite ignorance.” This picture misses something absolutely crucial, which is that we, as humans, are fundamentally part of the objective world—we are embedded in it, and even continuous with it, rather than irrevocably severed from it. Brian Cantwell Smith writes about transcending this metaphysical frame when he writes:

What would it be to recognize, finally, that the various forms of metaphysical separation—between representation and represented; between the parts of divvied-up reality; between and among concepts, types, or properties—what would it be to recognize that these forms of separation, like the separations we maintain in our political and emotional lives, are all partial: negotiated, gradual, welling up and subsiding, dynamically maintained, in a kind of on-going dance? What would it be to see the world as partially pulled apart, that is, making room for pluralism, error, autonomy, individuality, and heterogeneity, and as partially put together, making room for normativity, communion, humility, and transcendence?

It is not so difficult an image. Think of a potter pulling apart a particular sticky kind of clay, pushing globs of it away, stretching and squishing and clumping it together, forming shapes and drawing out spaces between and around it—except that we potters are just more clay.

Once we recognize that we are part of the world, constantly in contact with it and enmeshed in it—or as Cantwell-Smith puts it, “partially pulled apart” and “partially put together”—it becomes easier to accept the possibility of “direct observation” and “manifest truths.” There are many things you can directly observe, and which are “manifestly true” to you: what you’re wearing at the moment, which room of your house you’re in, whether the sun has set yet, whether you are running out of breath, whether your parents are alive, whether you feel a piercing pain in your back, whether you feel warmth in your palms—and so on and so forth. These are not perfectly certain absolute truths about reality, and there’s always more to know about them—but it is silly to claim that we have absolutely no claim on their truth either. I also think there are even such “obvious truths” in the realm of science—like the claim that the earth is not flat, that your body is made of cells, and that everyday objects follow predictable laws of motion.

As Cantwell-Smith puts it, our theories do partially mediate our access to the objective world—but they don’t entirely determine our perception of it:

There is much that is right in this argument [that we don’t have direct access to objective reality], especially the claim that we have no conscious access to the world independent of historical, cultural, and personal interpretation. But from the fact that we have no access independent of such interpretation, it does not follow that we have no access to it at all, or that the access we have is entirely determined by those interpretive schemes.

We are not disembodied explanatory souls trying to pierce into the nature of reality through a infinitely many miles of darkness and ignorance. We are in the world, part of the world, in direct contact with it. Our “direct access” to the world is a crucial ingredient to our success in obtaining knowledge—just as much as our capacity to create conjectures.

The triviality of “universal explainers”

One of Deutsch’s famous quips is that we are “universal explainers”: we, as humans, can explain everything that can be explained by any entity. Anything in the world that can be understood, can be understood by us. He uses this argument to claim, for example, that superintelligent AI couldn’t possibly be fundamentally different from us, and likewise with aliens. The difference between advanced AI and us would not be one of kind, but rather one of degree: whatever intelligence they have, we could supplement our own intelligence with more memory and faster processing speeds (e.g. with computers, or trivially, notebooks and pencils) in order to get to the same level of intelligence.

Dwarkesh Patel writes about why this is a silly argument:

This is a bit like saying that it’s inaccurate to suggest that a car has a higher range or speed than a bicycle. With future technology, we could replace the bicycle rider’s ATP stores with an atomic battery, and its muscles with miniature jet engines. In fact, bicycles and cars are both “universal vehicles” which share the same fundamental capacity for transportation. The only differences which can exist between them are the size of their energy supply and the power of their engines. Since you can always extend these two attributes, one vehicle cannot be more powerful than another.

In Deutsch’s view, our brains are basically Turing machines but with a special “creativity algorithm” installed—that’s what distinguishes us from other creatures. A Turing machine can always be augmented with more memory and faster processing, so we are functionally equivalent to aliens and superintelligent AI’s. This is technically true, and functionally unhelpful. Those who are concerned about AI safety are worried about AI systems becoming much more intelligent and more competent than us very quickly. If these systems do surpass us that quickly, it doesn’t really matter that we could technically augment our brains with a bunch of memory—we become functionally much less capable than them.

Deutsch’s claim that we are universal explainers involves a highly speculative connection he draws between the Church-Turing-Deutsch principle (which asserts that a sufficiently powerful computer can simulate every physical process) and our own brain’s capacity to understand the world. It assumes that when we form explanations, we are “running a program that simulates the outside world” in the exact same way that a computer simulates weather patterns. This seems plausible, but is not a given – many people disagree with this framing, and in the absence of a clear understanding of how our brain forms explanations, it’s hard to settle the question.

Beyond this tenuous connection, Deutsch’s point about universal explainers basically boils down to an instrumental argument. The argument for universal explainers is: if there are things we truly can’t understand, then by definition we can’t talk about them (otherwise we’re basically “talking about the supernatural”), so why bother talking about them. It’s an argument of the form, “if I can’t see it, it doesn’t exist”—which is a decent enough argument in practice. Except that it’s an instrumental argument, and in most other situations, critical rationalists staunchly oppose instrumentalism.

False predictions on AI

In his 2012 essay Creative Blocks, Deutsch argued that the dominant AI paradigm was doomed, because it was ignoring the key capability of humans: our ability to create and criticize conjectures. He was convinced that the approach of building AI by “learning from experience” would never work, because we as humans don’t “learn from experience.” This is part of Deutsch and Popper’s critique of empiricism—the view that knowledge is derived from our sense experiences. In Deutsch’s world, theories always come first. We formulate a theory (perhaps consciously or unconsciously), and that theory is the framework through which we make observations. The observations might refute our theory, in which case we discard it and come up with a new theory.

The predominant paradigm in AI for the past few decades has been the opposite: start with a bunch of data, and train your algorithm to figure out patterns in the data, and hope that it will generalize to uncover “deeper knowledge” contained in the data. Many people were convinced this wouldn’t work at first, that merely extrapolating from statistical patterns in data would never amount to true intelligence. And then, year after year, this exact approach in AI helped to solve problems that we had previously been unable to get computers to solve, and we now know it’s enough to pass the Turing test. The very approach of “learning from data” which critical rationalists would deride as “empiricist” can now outperform doctors at diagnosing illness and support graduate-level math research, among many other capabilities. But because this AI is built under an “empiricist” paradigm, critical rationalists still insist that they are not and will never be intelligent in the way we are.

Deutsch believes that everything that is special about humans comes down to the “creativity algorithm” that’s implemented somewhere in our brains. This is the thing that enables us to make conjectures about the world, and Deutsch has speculated that it’s also the same thing that gives us free will and consciousness. And on top of that, he asserts that this is the same algorithm that any other “generally intelligent entity” in the universe must have. To be fair, the questions of intelligence, consciousness, and free will are all open questions, and Deutsch’s idea of the “creativity algorithm” is intriguing and worth exploring—but at this point it seems too simplistic to pin down every big question about our humanity to a single, discrete algorithm in our brains that we just haven’t found yet. Many other creatures seem to be conscious even if they don’t make “conjectures” about the world, there are plenty of examples of intelligence in the absence of “explanatory knowledge”, and there are now machines that convincingly speak and think, but who never had a creativity algorithm installed in them. I would not bet on all of these capabilities coming from the exact same place, in a single “jump to universality” that we made.

An intolerance of ambiguity

There is one other aspect of the Deutschian frame that Popper and Deutsch never seriously question: it’s that the correct way to think about “how we obtain knowledge of the world” is through the lens of formal logic.

The title of Popper’s first book is A Logic of Scientific Discovery—he was specifically after a logic of science, and that quest has been continued by Deutsch. The critical rationalists think about science in the frame of “forming logical statements,” and figuring out the truth or falsity of those statements.

The problem with formal logic is that it is intolerant to ambiguity, and it can’t accommodate any notion of “partial truth.” If any part of a system of logical statements is untrue, its falsehood “bleeds out” into the entirety of the system. This is why critical rationalists make statements like “Newton’s laws are false”—because for as much as we’ve gotten out of Newton’s laws, the fact that we’ve found conditions in which they do not apply makes the theory as a whole strictly false. And this is also why it’s impossible, under fallibilism, to ever establish that any theory of ours is true—because we are concerned with logical truth, and logical truth requires us to check every conceivable instance and implication of a statement, which is completely impossible.

(Of course, the more commonsense view of science is that we are not after strict, perfect truths, and so it makes no sense to call Newton’s laws “false”—they are true in many circumstances, and false in others. Like all theories, they have domains in which they apply, and domains in which they don’t. The fact that they are imperfect does not mean that we’ve discarded them entirely.)

In a conversation on statements, propositions, and truth, Deutsch makes his metaphysical assumptions explicit when he says that “reality is pristine”—in his view there is a perfectly precise truth of the matter to every aspect of reality. Not that we’ll ever reach it, but it’s there.

But we have very little reason to think that “reality is pristine”—what we’ve found again and again is that the world is not amenable to exact, perfect separation into well-defined categories. Hofstadter illustrates how the concepts we use to slice up the world all ultimately have poorly defined boundaries, and David Chapman does the same for everyday objects. The aforementioned Brian Cantwell-Smith paper likewise refutes this stubborn “metaphysical discreteness.” Franklin and Graesser make the same point when they comment offhand that “the only concepts that yield sharp edge categories are mathematical concepts, and they succeed only because they are content free.”

For the critical rationalists, all this ambiguity and fuzziness that we find in our concepts is simply a matter of our own fallibility—there is a perfectly precise truth of the matter about everything in the world, we just have trouble seeing it because of our limited perspective. According to Deutsch, there is a collection of “infinitely precise and perfectly unambiguous abstract propositions” which describe every single facet of the universe in complete detail. These propositions exist in the abstract realm, independent of space and time. Now, it’s impossible to say whether these propositions really do exist, but what’s clearer is that we really don’t need them, if we stop framing our quest for knowledge as a search for logically true propositions (or approximations to such propositions).

I said earlier that the critical rationalist project will ultimately be deemed a failure, and this is the exact sense in which I mean it: we will eventually give up on trying to formulate science and knowledge-creation as approximations to formal logic. We already have plenty of reasons to believe this project is doomed—whether it’s in the intractability of the frame problem and relevance realization, or the pesky fact that semantics cannot be reduced to syntax. And it’s not a problem: we carry on creating new knowledge regardless. What I suspect will happen is that we will, collectively, “set aside” this project, in the same way that we’ve set aside previous philosophical projects, as Richard Rorty describes in Philosophy and The Mirror of Nature (emphasis mine):

Wittgenstein, Heidegger, and Dewey are in agreement that the notion of knowledge as accurate representation, made possible by special mental processes, and intelligible through a general theory of representation, needs to be abandoned. For all three, the notions of "foundations of knowledge" and of philosophy as revolving around the Cartesian attempt to answer the epistemological skeptic are set aside. Further, they set aside the notion of "the mind" common to Descartes, Locke, and Kant—as a special subject of study, located in inner space, containing elements or processes which make knowledge possible. This is not to say that they have alternative "theories of knowledge" or "philosophies of mind." They set aside epistemology and metaphysics as possible disciplines. I say "set aside" rather than "argue against" because their attitude toward the traditional problematic is like the attitude of seventeenth­ century philosophers toward the scholastic problematic. They do not devote themselves to discovering false proposi­tions or bad arguments in the works of their predecessors (though they occasionally do that too). Rather, they glimpse the possibility of a form of intellectual life in which the vocabulary of philosophical reflection inherited from the seventeenth century would seem as pointless as the thirteenth-century philosophical vocabulary had seemed to the Enlightenment. [...] Wittgenstein, Heidegger, and Dewey have brought us into a period of “revolutionary” philosophy (in the sense of Kuhn's “revolutionary” science) by introducing new maps of the terrain (viz., of the whole panorama of human activities) which simply do not include those features which previously seemed to dominate.

We can put aside the Deutschian project. Rather than trying to eradicate ambiguity and keep seeking “perfect truths,” we can embrace the inherent nebulosity in the world, and accept that exactness and ambiguity are two inseparable elements of the world and our relationship to it.

III. Systematic mistakes in Deutschian thinking

With these last few sections, what I’d like to do is take a more meta/sociological stance towards critical rationalism, where I try to tackle why it’s possible for someone to be as confused as I claim that critical rationalism is, while also being staunchly defensive of it.

Popperians ignore the problem of meaning to their peril

This isn’t so much an obvious logical problem with critical rationalism, but rather a way of thinking that leads a lot of critical rationalists into confusion.

Popper famously described the problem of meaning as a “pseudoproblem” – as long as we think we understand our counterpart’s view, we don’t need to obsess over the meaning of words. He wrote, for example:

Linguistic precision is a phantom, and problems connected with the meaning or definition of words are unimportant. Words are significant only as instruments for the formulation of theories, and verbal problems are tiresome: they should be avoided at all cost. (37-38)

Popper is right that it’s possible to focus too much on the definitions of words, but in his broad dismissal he also ignores genuine problems about the relationship between our ideas, our language, and the world. The meaning of words is neither inherent to the shape of the words themselves nor to our brains – it lies in our interaction with the world, as David Chapman illustrates with the parable of the pebbles:

In the 1980s, this conundrum, “the problem of intentionality,” became the central issue in the philosophy of mind. Cognitivism collapsed when it became clear that no answer is possible—not because we don’t know enough details about how brains work, but even in principle.

Representation is not a property of the bucket, pebbles, or sheep. It’s a property of the whole history of interaction of the bucket, pebbles, shepherd, sheep, and gate. Likewise, beliefs aren’t in your head; they too are dynamics of interaction. Representation can’t be found in a snapshot of the state of the world, nor in a timeline of brain activity. It’s necessarily a process extended in both time and space.

Why does this matter for critical rationalism? This bias against problems of meaning leads some critical rationalists to never question the meaning of their utterances, even their most abstract philosophical claims, because they’ve internalized the idea that investigating meaning is a waste of time. A common pattern in critical rationalist thinking is to define words in very particular ways, and then refuse to question those definitions. Deutsch finishes each chapter of his books The Fabric of Reality and The Beginning of Infinity with a list of such definitions. Sometimes the definitions are interesting reframings of everyday concepts (like defining a “person” as “an entity that can create explanatory knowledge”), but other times they are caricatures of opposing positions that are specifically framed in a way that bolsters his argument.4 It’s fine to define words in new and interesting ways, but the denigration of “problems of meaning” leads some critical rationalists into a blindspot, where they imagine that their disagreements with other philosophers are always about substance and never about mere meanings, and fail to see the way they and their opponents are talking past each other.

Popper dismisses introspection

Another common tendency among critical rationalists is a dismissal of introspection as a useful tool in doing philosophy. This happens because a lot of critical rationalists have internalized a strict distinction between the logical content of our ideas (which, in their view, is the only thing that really matters for epistemology), and our psychological states, like the emotions we have about those ideas. Popper does this explicitly in the first chapter of The Logic of Scientific Discovery, in a section literally titled “elimination of psychologism.”

I now believe that cleanly separating the content of our ideas and our feelings about our ideas is extremely difficult if not outright impossible, and attempting to do so only makes it harder to think clearly. Raymond Smullyan’s essay Is God a Taoist? is a good example of how introspecting a bit can help partly resolve abstract philosophical problems: he shows that much of the confusion about free will stems from an implicit analogy we make between “the laws of physics” and “laws” in the sense of civil code:

Your acts are certainly in accordance with the laws of nature, but to say they are determined by the laws of nature creates a totally misleading psychological image which is that your will could somehow be in conflict with the laws of nature and that the latter is somehow more powerful than you, and could "determine" your acts whether you liked it or not. But it is simply impossible for your will to ever conflict with natural law. You and natural law are really one and the same. […]

Don't you see that the so-called "laws of nature" are nothing more than a description of how in fact you and other beings do act? They are merely a description of how you act, not a prescription of of how you should act, not a power or force which compels or determines your acts.

Here’s a specific example: I think many of the biggest fans of critical rationalism would be well-served by introspecting on what they find so appealing about David Deutsch’s books. If you’re not careful, a book like The Beginning of Infinity, despite all its claims about how everything is conjectural, can ultimately become a kind of “existential grounding” for you, psychologically. You can test this by asking a simple question: if you found out that Deutsch was deeply wrong – like, stupidly and obviously wrong, about many important points – would you have a negative emotional reaction to this discovery? If that's not the case for you – then great, you are not emotionally attached to Deutsch's ideas. But I imagine there are at least a few people out there who are fans of Deutsch specifically because of the sense of meaning it gives them, as it did for me.

This is especially true if you’re someone who has a soft spot for philosophy, for big ideas. And in particular, a soft spot for the power of thought. You have always kind of wished that, by virtue of just sitting in a room with your books and pencil and paper, and just sitting down and thinking hard about things, you could potentially land at a truly transformative, earth-shaking insight. And not only would it be a powerful insight, but it would be an intuitively understandable insight – an explanation. It would upset you if this is not possible. Deutsch’s work is emotionally satisfying because it puts theorizing at the very center of human activity, as the most significant and consequential thing that we humans do.

To be clear, this entire argument on its own is not a refutation of Deutsch’s ideas. Rather, it’s a nudge: that if you’re someone for whom all of the above psychological points apply, then you might be well served by inspecting whether some of your defenses of Deutsch’s ideas are genuine arguments you believe, or if they stem from a desire to rationalize a worldview you’re emotionally attached to.

Critical rationalists blind themselves to the ladder of abstraction

In a podcast discussion between Jake Orthwein and Chris Lovgren, they point out one of the funny quibbles between fans of Popper and fans of David Chapman: both parties criticize the other party as being “too abstract.” When I first encountered this claim, I actually had difficulty understanding how critical rationalism is abstract. And I think at least part of the answer is that the critical rationalist worldview blinds you from seeing just how abstract it is.

In the critical rationalist worldview, everything in your mind is an idea. Feelings are ideas, perceptions are ideas, thoughts are ideas. Every piece of knowledge we have is made up of ideas, or theories. And as I mentioned in the section on “observation is theory-laden,” all of our observations are also inextricably enmeshed with ideas. Deutsch also argues forcefully for “the reality of abstractions” (one of the chapters in his book), asserting that any idea that plays a role in our “best explanation” of something is a real thing. In effect, he says that highly abstract concepts like complex numbers, causality, and counterfactuals are just as real as the table in front of you. Your understanding of the table, after all, is mediated by a bunch of abstractions! We exist in a thick morass of abstractions and there is simply no way out of it.

Once again, we’re in a situation where what Deutsch claims is true in a technical sense, but it distorts your understanding of the world in a profound way. In the more commonsense view of the world, there is a very obvious distinction between “abstract” things and “concrete” things. What makes something concrete is that it’s here, now, tangible. What makes something abstract is that it requires layers of concepts to talk about: to talk about complex numbers, you first need to talk about the abstract notion of “real numbers”, which are an abstraction built on top of “whole numbers,” which are still abstractions over the concrete notion of “two apples in front of you.” There is an important sense in which the “concrete” things are “more real” than the “abstract” things—as in, given our particular brains and bodies, they are easier to grasp and their existence is more easily verifiable.

When you enmesh yourself deeply in the Deutschian worldview, this hierarchy is wiped away.5 The notion of “complex numbers” and that of “two apples in front of you” somehow end up on the same footing, because they are both theory-laden and conjectural. This is what makes it so that critical rationalists have trouble seeing why their claims are so abstract, and as a consequence so frustrating to argue with. (They’ll make statements like “Popper proved that theories are not generalizations of observations,” which is only true when you take on their particular, strained definitions of “theory” and “generalization,” but is otherwise a hopelessly vague claim.6)

This brings us to another, related hierarchy that critical rationalists blind themselves to:

Critical rationalists dismiss the “hierarchy of reliability”

Most people subscribe to the commonsense view that there’s a certain “hierarchy of reliability” between math, science, and philosophy, where mathematical truths are taken to be “certain,” scientific truths are “likely true, or strongly supported by evidence”, and philosophical truths are “largely a matter of taste.” In an example of this way of thinking, Erik Hoel writes that “Most disagreements in a highly abstract field break down into quibbling over language. Wittgenstein showed us that.”

Deutsch claims that this intuition is incorrect, because all of our knowledge is conjectural, and so we are equally unsure about everything. For example, he says that our knowledge of solipsism being false (the claim that there is nothing outside your individual mind) is equally as compelling as our knowledge that the square root of 2 is an irrational number. The claim that “knowledge is created via conjecture and refutation” is equally as compelling as the claim that “the earth is not flat.”

Deutsch writes:

Some philosophical arguments, including the argument against solipsism, are far more compelling than any scientific argument. Indeed, every scientific argument assumes the falsity not only of solipsism, but also of other philosophical theories including any number of variants of solipsism that might contradict specific parts of the scientific argument.

There are two different mistakes happening here.

First, what Deutsch is doing is assuming a strict logical dependency between any one piece of our knowledge and every other piece of it. He says that our knowledge of science (say, of astrophysics) implicitly relies on other philosophical arguments about solipsism, epistemology, and metaphysics. But anyone who has thought about the difference between philosophy and science recognizes that in practice they can be studied and argued about independently. We can make progress on our understanding of celestial mechanics without making any crucial assumption about metaphysics. We can make progress studying neurons without solving the hard problem of consciousness or the question of free will.

The second, bigger problem with Deutsch’s claim is that, like many of the other points in this essay, it’s true in a technical sense but completely misses the point. Deutsch is right that ultimately, we are not certain of mathematical truths, just as we are not really certain of anything.7 But here he’s using a literally impossible standard of “certainty,” where it means “known without any possible sliver of doubt and there is no chance in any conceivable world where this turns out to be false in the future.”

A more commonsense definition of certainty might be: we’ve thought long and hard about it, and we have extreme difficulty seeing how it could possibly be otherwise. In this way, we are certain of mathematical truths, less certain of many scientific truths, and completely uncertain about philosophical truths. This is obvious in everything from the way we talk about progress in math versus science and philosophy (we talk about “proofs” in math, compared to “arguments” in philosophy), to the rate at which we resolve disagreements (philosophical debates tend to take hundreds of years to resolve, and even then, are only ever really resolved once the question becomes concrete enough to become part of science).

To put it in terms that might make more sense to a critical rationalist: while our knowledge in all these fields is conjectural, we have wildly different methods of criticism for the ideas in them, and some of those methods are more robust than others. It’s very easy to come to agreement about the validity of mathematical proof (so much so that we have literally programs that do this mechanically), whereas philosophical arguments can be litigated endlessly because they are abstract and very imprecise. Science occupies a middle ground, where its claims are concrete enough that they can be tested and falsified with experiments, and it’s easy to agree on the outcomes of experiments.

When one twitter user once asked why so many critical rationalists are perceived as dogmatic, Brett Hall responded that it’s because people subscribe to this false hierarchy of reliability, and “no one accuses a mathematician of being dogmatic when they prove a theorem.” My claim is that it takes a particular kind of mental straining to defend yourself against claims of dogmatism by saying that “no one talks about mathematicians as dogmatic”—it betrays a belief that your philosophy has as much argumentative force as a literal mathematical proof.

IV. Coda

The problem with critical rationalism is not that it’s wrong in one fundamental way – but that it’s right in a bunch of unimportant ways. How many scientists actually use the idea that “beliefs don’t exist” to do their work? Who relies day-to-day on the maxim that “we don’t know if anything is true or likely to be true, only that it’s not yet falsified,” other than conspiracy theorists and self-help gurus? In response to this, Deutschians would say: the fact that these ideas are not widely used says nothing about how true they are. And that is, once again, strictly true – but it misses the point. You’ve backed yourself into a conceptual corner where all you’re doing is telling convenient stories about what the world is like, and no one’s really going to disagree with you, because your stories are “not even wrong.” That is why I call it the “Deutschian deadend.”

Critical rationalism does have some nuggets of truth in it, and thus it has value as one of many ways of trying to understand our relationship to the world—but it’s certainly not “the best explanation” for how we obtain knowledge. If you ask cognitive scientists, AI researchers, linguists, neuroscientists, and psychologists – do any of them credit critical rationalism and Karl Popper with giving them foundational insights into how our minds work and how we learn about the world? They don’t. Of course, many critical rationalists will then assert that this is why All Of Those Fields Are Wrong, and if everybody just sat down and understood science and knowledge in the exact way that we do we would no longer have this problem.8

To close, there are two sociological observations I’d like to make that, while not directly refuting this worldview, help illustrate the flaws in it. First, critical rationalists are staunchly anti-dogma: Popper’s whole schtick was that we are fallible beings, constantly prone to error, and so we should never hold any of our ideas as immune to criticism. And yet, critical rationalists are often described as more dogmatic than their counterparts. In a second contradiction, critical rationalists are emphatically against focusing too much on the meanings and definitions of words—and yet in many discussions, they seem particularly fixated on word choice and specific definitions.9 The point is: there are multiple contradictions between some critical rationalists’ stated beliefs (anti-dogma, against fixating on definitions) and their actions. And in my view this is yet another meta-problem that points to flaws within the philosophy itself.10

The reason this essay was important for me to write is that I was once the kind of person I’m describing here. After I read a lot of Deutsch and Popper, I started viewing most other philosophers of science as fundamentally confused, and even much of commonplace thinking about epistemology as mistaken, and made lots of bold pronouncements on twitter to that effect. I got into debates with friends about how their commonsense intuitions are wrong, and this new idea is actually the correct epistemology and is one of the most important ideas ever articulated. But then, over the course of another few years, more reading and thinking, and many conversations with friends who had gone through a similar evolution, I began to see the way that Popperian thinking rationalizes any critique as a form of misunderstanding, and ultimately makes its adherents parrot-like in their utterances. I eventually realized that all those principles I cherished—human fallibility, the possibility of understanding the world better despite it, and our potential to transform the world—are still true, even without all these tortured rationalizations.

Thanks to Ben and Jake for feedback on earlier drafts.

The story in brief: a guy walks up on a hill, and suddenly he experiences himself leaving his body, and zooming out into outer space, and moving faster and faster away from the earth at the speed of light, and he begins a telepathic journey across the entire history of the cosmos. He travels with his mind, and “merges” into the mind of creatures throughout the cosmos, experiencing life as them. He meets other people who are also traveling across the universe, hopping between different minds, and they become a traveling circus, and the group becomes larger and larger over time, and in the end he zooms out to become the entire universe itself.

One theme of the book is the “expansion” of consciousness: he starts as a human mind, and then enters the mind of other human-like creatures on other planets and galaxies, but soon he discovers that there are larger entities that can have a mind of their own. In more advanced societies, individual creatures bind together to form a “collective mind”, and these collective minds can span entire planets, and eventually entire galaxies. Later in the book he also discovers that there are minds in places we don’t expect them: galactic nebulae have primitive minds of their own, and stars have slightly more sophisticated minds.

The exploration of alien civilizations was quite interesting. It’s a kind of David Attenborough documentary on aliens. He talks about these creatures that are a hybrid of plants and animals: they move around at night like animals, and during the day they stand still and absorb the light of the sun, experiencing a kind of religious ecstasy in their communion with the light every day. He discovers flying creatures on another planet which have primitive minds individually, but when flocking together develop higher-level mental faculties like intelligence.

(Unfortunately the book enumerates too many examples of different of alien species, becoming frustratingly repetitive. It definitely could have been cut down to a tenth of its length and made into a great short story.)

But what I found most interesting about the book was a very clear message that the author was making: spiritual transcendence is a common cause across all living beings in the universe, and is baked into the nature of the universe itself. As the narrator was exploring the various civilizations of different degrees of advancement, he kept pointing out the distinction between material progress and spiritual progress, and how ultimately all civilizations were striving to advance both. Material progress is the ability to manipulate the physical world to suit certain ends. Spiritual progress is the ability to see the world more clearly, and to discern which ends are worth pursuing.

Reading all of this gave me a sense of “cosmic religion.” I mean that in two ways: first there’s the simple feeling of awe that you get from appreciating the scale of the cosmos, and Stapledon really pounds that into you throughout the book, by talking for example about planets that dislodge from their star system to engage in a cross-galaxy expedition. The universe is massive, and simply appreciating this brings one into a kind of religious trance. It’s the kind of feeling you get from watching Dune 2.

But there is a second sense of “cosmic religion” the book made me think about. Reading Star Maker has made me more optimistic that we can actually savor the benefits of organized religion while at the same time maintaining an undogmatic, scientifically-driven worldview.

Why do we need religion?

First let me lay out my cards on how I think about religion as a whole. I’m not religious (except for briefly when I was a child), and for much of my life I was staunchly anti-religion. The word “religion” had a primarily negative connotation as I was growing up: it meant orthodoxy, oppression, zealotry, and ignorance. Today I still think all of those things can be true of religion, but I’m also more sympathetic to its benefits.

Today the word “religion” really just means to me: a shared, meaningful story that puts our existence as individuals into a broader context. Not just any broad context, but the broadest possible context. Religion conceptualizes our existence as individuals into a meaningful story about the entire universe.

We already have lots of meaningful stories that contextualize our lives into more medium-sized contexts. Patriotic Americans find meaning in their citizenship of a liberal, constitutional democracy that espouses individual freedom and pluralism. The employees of OpenAI feel a shared sense of purpose in their quest to build AGI and usher in a new age of prosperity. You might find a sense of meaning as a member of a friend group, a club, or a family.

Some would argue that these “smaller-scale” sources of meaning are enough. They give us a role to play in the world and a community to connect with. But something is missing from this picture, for a few reasons:

• These local meanings are highly contingent. If your life has meaning because you work for OpenAI, what if you get fired? What if your treasured friend group disbands because everyone moves to different cities? Maybe you feel a sense of meaning from being an American, but what happens if your country turns into something unrecognizable, and moves in a direction you believe to be fundamentally un-American?

• These local meanings don’t help us cooperate in larger and larger groups. Like, we’re good at getting a company of five hundred people to work together, and perhaps we can even get a country of three hundred million to work together (just barely), but what about the entire earth? The world is more peaceful today than it has been for most of modern history—which is indicative at least of one globally shared value: we all want peace, more than we want nuclear armageddon—and yet there are still severe conflicts raging, both in terms of actual wars (the Middle East, Ukraine) and also in terms of inter-country tensions that could easily spiral into larger wars (US vs China, Europe vs Russia).

• The local meanings also don’t give us enough of a shared language around the biggest questions in life, specifically for those who aren’t part of an organized religion. What happens when we die, and how should we honor those who die? How should we express our sympathy for those who are doing poorly, and in what way can we wish them well? Traditional religions have answers to these questions: the afterlife, funeral services, prayer, and communion. But for those of us who aren’t part of a religion, we’re kind of left with a patchwork of rituals and sayings (”keeping you in my thoughts”), which are better than nothing, but which are severely lacking compared to the kind of shared rituals we had in religion.

To get a sense of what I mean about “a shared language,” see this video. In it, social media influencer Bryce Crawford runs into a homeless man. The homeless man is at first threatening and asks for money, but Bryce quickly disarms him. Bryce says to the man: “I love you. When was the last time someone told you that?” The man, obviously taken aback and emotional, says, “pray for me.” And they then huddle closely, and Bryce says a prayer for the man. “Father lord, would you fill him with strength; God comfort him right now. Would you bless him, keep him safe, reveal yourself to him in dreams and visions.” It’s a very touching and humanizing moment, even without the prayer. But I think the prayer—and the shared sense of meaning that both Bryce and the man take in the prayer—adds a deeper level of salience to the encounter. I would bet that, as an empirical fact, that moment of prayer left the homeless man psychologically much better off than had it not happened. It will make him just slightly more resilient in future moments of need. Prayer can empower the people who do it even without any magical or metaphysical effects. Prayer can be as simple as a shared intention for a better life, expressed in a shared language, and which connects our individual desires for a better life into a bigger story about goodness and the human quest for happiness.

What would a cosmic religion look like?

Now that I’ve made the case that something like religion could help us today, we have the question of what this “cosmic religion” would actually be. I have fewer answers here, but I think Star Maker gives us an interesting starting point for discussion. All of the following is obviously biased by my own values and worldview.

Just as we can (and have) attained higher and higher levels of material and cognitive attainment over the centuries, we can also attain higher levels of moral and spiritual attainment.

In the material world the signs of progress have been obvious: we have built larger and larger buildings, we’ve edited DNA, we’ve isolated the building blocks of atoms, we’ve built computers that speak and think, we’ve built rockets that fly into space and that even land safely back on the earth. We’ve achieved worldwide declines in poverty, we’ve connected the entire world into a shared web of information flow, and we’ve optimized agriculture to the point that only a small fraction of humans work on farms to feed the entire world’s population.

We’ve also developed cognitively: literacy is near ubiquitous in most countries, we’ve developed sophisticated theories of stars, matter, cells, and brains, we’ve created new fields of mathematics, we’ve dissected human language, we’ve invented countless algorithms and data structures for manipulating information, we’ve created more and more complex fiction, and we’ve become smarter on average with each generation.

What does moral and spiritual progress look like? We’ve also made significant progress here. For example, over the centuries we’ve dramatically expanded the circle of “who has moral worth,” to include women, people of color, sexual and gender minorities, and we’re continuing to increase that circle over time. Unlike the other markers of progress, though, spiritual progress has stagnated and in some ways even reversed in the past few hundred years. An obvious marker of this: depression and suicide are on the rise, we have epidemics of drug addiction, and the fixation on “mental health” seems to be higher than ever.1

Here are some ways that we could make more spiritual progress:

• Broadening our level of compassion for everyone. This includes people we believe to be “worse off” than us, and also people who we believe are “better off” than us. It includes people we think are “good,” and people we think are “bad.” True spiritual progress looks like an ability to feel compassion for every person and every being, even while we might hold negative judgements about their character or actions.

• Increasing our “lucidity.” Being less consumed by prejudice, less consumed by self-limiting beliefs, less consumed by stories that create suffering rather than alleviating it. Being more aware, in each moment, of what’s actually in front of us, and what we’re imposing onto our experience via our conceptual interpretations.

• Resolving more and more disputes with reason rather than violence. The hallmark of a civilized society is that disputes are settled by willful cooperation between different parties, based on mutual trust, rather than the threat of violence. In the past, if you and someone else disagreed about something you would have a duel to the death; today, you usually resolve your disagreement by talking about it.

• Deeper and deeper understanding of the cosmos. “Understanding” here takes many forms: it includes conceptual understanding (for example, probing into the laws of physics, the origin of life, the nature of cognition and consciousness), but also experiential understanding (a “mastery” of the mind as in meditation, “awakening” in the Buddhist sense), and even aesthetic understanding (a deeper appreciation for beauty in a variety of forms), among others.

These are the basic tenets that cosmic religion could be oriented around. It could be more of a meta-religion, establishing a core set of values but open to being implemented in a number of different concrete rites and rituals by different groups as they see fit.

Cosmic purpose

There was a particular passage in Star Maker that stood out to me. The narrator talks about how at one stage in the history of the universe, many stars began to die out, and this compelled the inhabitants of those star systems to seek out new stars. And so what they would do is launch their entire planet out of the orbit of their home star and begin a long journey to another star in the galaxy. And of course, such a journey, even with advanced space-faring technology, would take many thousands of generations (during which the planet is supported by a temporary “artificial sun”). The point of the story is this: there is a deep sense of shared responsibility between each generation on the planet with all of its past and future generations. To put it to a finer point, imagine that you’re on an intergenerational mission in space, with a well-defined starting point and destination (say, a far away planet that can harbor new life). Your role in that mission is extremely well-defined, and for the modern optionality-obsessed young person, this can be exhilarating. You have a clear duty to continue the mission, and the entire fate of both your ancestors and descendants depends on it.

Now, this isn’t necessarily a situation that all of us want to or should be in. In an ideal world society is resilient and sophisticated enough that the entire fate of all future beings doesn’t rest in a single person’s hand. But at the same time, there’s an important perspective to take away from this story. It’s that in some deep sense, you really are playing a causal role in a vast cosmic history made of billions and billions of small decisions, which ultimately do add up to some sort of bigger picture, which do ripple out into the experiences of billions and perhaps trillions of beings in the deep future. This is something that I think is already true whether we see it or not, but something like a “cosmic religion” can help us see it more clearly, more of the time.

Reading Star Maker has left me with more questions than answers. How will humanity’s religiosity express itself over the next few decades and centuries? Are there truly universal values that we can all agree to cultivate in ourselves and our societies? Will we ever converge on a shared story for our role in the cosmos, and what will the point of that story be? For the sake of our posterity, these are questions worth asking.

Thanks to Suzanne for discussion and feedback.

What happens in successful therapy? You talk to someone, have many hourlong conversations, and perhaps you cry a bunch—but what is actually happening that leads to you feeling better (and sometimes worse)?

Surprisingly, we don’t have a widely agreed-upon answer to this question. This is the study of “mechanisms of change” in therapy, and it’s something that psychologists (and more recently, neuroscientists) have been going back and forth about for over a hundred years. Some argue that the core mechanism of change in therapy is memory reconsolidation, in which you are updating past memories of traumatic experiences. Others argue that the core function of therapy is not to re-adjust traumatic memories, but to create new memories that compete with the older memories when determining behavior. Still other people frame psychological change primarily in terms of constructing and deconstructing personal narratives, or in terms of processing and better relating to your emotions.

This is where psychotherapists Bruce Ecker, Robin Ticic, and Laurel Hulley came into the picture and said: we know the fundamental process that takes place in successful therapy, across all the different therapy modalities, and we have the neuroscience to back it up. They wrote Unlocking the Emotional Brain (UTEB) to explain their theory. It’s a very bold agenda, and regardless of whether it ultimately succeeds, the book has had a fundamental impact on how I think about human psychology, and I think it can help everyone understand themselves better.

Subconscious beliefs dictate behavior

Here’s the gist of the theory behind UTEB. Inside our mind is a bunch of “schemas”—mental models we’ve formed of the world based on past experiences. Ecker and team believe that much of our behavior is dictated by these unconscious, emotional models of the world that we may have learned years in the past. For example, if you’re conflict-avoidant, the authors would argue there’s some implicit belief that drives that behavior: maybe you harbor a belief of the form “if I upset someone, they’ll hold a grudge against me forever.” Now, this might not be a conscious belief, but the idea is that this mental model is well-formed in your subconscious and guides your behavior without you even noticing it.

Unlike other therapies that might focus especially strongly on the past experiences that led to this belief, or unprocessed emotions that guide the belief, the authors of UTEB are primarily interested in the mental model itself, and how to uproot it. Any traumatic memories (say, of someone holding a grudge against you) might help you identify the problematic model, but the focus is on updating the mental model today, rather than marinating in the past. The authors believe that all successful therapy is ultimately about uprooting beliefs, and doing so in a visceral and decisive fashion.

Case study: anxiety at work

To help understand what it means to “uproot unconscious beliefs,” it’ll help to look at an actual case study of the process.

Consider the case of Richard.1 Richard had paralyzing anxiety at work, particularly in the context of contributing to discussions. He was constantly blocked from speaking up in meetings, despite generally receiving positive feedback for his contributions. In order to uncover the “underlying emotional belief” that was causing this behavior, the therapist engaged in a visualization exercise with him, in which Richard was asked to imagine contributing to a team discussion without inhibition. When Richard imagined contributing to a meeting, this happened:

Richard: [Pause.] Now they hate me.

Therapist: “Now they hate me.” Good. Keep going: See if this really uncomfortable feeling can also tell you why they hate you now.

Richard: [Pause.] Hnh. Wow. It’s because… now I’m… an arrogant asshole… like my father… a totally self-centered, totally insensitive know-it-all.

Therapist: Do you mean that having a feeling of confidence as you speak turns you into an arrogant asshole, like Dad?

Richard: Yeah, exactly. Wow.

Now the therapist has helped Richard to identify the relevant “schema” in his mind: his anxiety is based on the belief that if he spoke up at work, he’d be hated. How do we update this mental model? The authors stress that you don’t just want to say: stop thinking that way. This is roughly the approach used in “counteractive” therapies like cognitive behavioral therapy (CBT), and it can work, but is harder to carry out for deeply held emotional beliefs. Instead, you want to treat the belief as valid and plausible to start (it had an adaptive purpose, after all), and then you want to present the mind with definitive evidence for why it’s not true.2 You want to update the belief at an emotional level, rather than just an intellectual one.3

The usual way to do this is to induce a strong juxtaposition experience between the original belief and some experience that violates that belief. They had Richard do two things simultaneously: (1) bring to mind his intense anxiety at work, and (2) recall a recent experience in which he observed someone else contribute to a meeting without being hated or ridiculed.

Therapist: Mm-hm. [Silence for about 20 seconds.] So, how is it for you be in touch with both of these knowings, the old one telling you that anything said with confidence means being like Dad, and the new one that knows you can be confident in a way that feels okay to people?

Richard: It’s sort of weird. It’s like there’s this part of the world that I didn’t notice before, even though it’s been right there.

Therapist: I’m intrigued by how you put that. Sounds like a significant shift for you.

Richard: Yeah, it is. Huh.

Therapist: You’re seeing both now, the old part of the world and this other part of the world that’s new, even though it was right there all along. So, keep seeing both, the old part and the new part, when you open your eyes in a few seconds and come back into the room with me. [Richard soon opens his eyes and blinks a few times.] Can you keep seeing both?

Richard: Yeah.

Therapist: What’s it like to see both and feel both now?

Richard: [Pause, then sudden, gleeful laughter.] It’s kind of funny! Like, what? How could I think that?

By virtue of this visceral, emotional experience of the conflict between the belief that (1) “If I speak confidently people will hate me”, and (2) “when others speak confidently they are not hated”, Richard effectively rewired the part of his brain that held the first belief. The authors describe how after a few repetitions of this “juxtaposition experience,” Richard came back in future sessions and reported the total absence of his prior anxiety, to his immense relief.

Transformational change vs incremental change

One of the key theses of UTEB is that there are two categories of outcomes in therapy: transformational change and incremental change. Transformational change looks like the case above: a small number of sessions that target the root of an undesired symptom, and bring about lasting resolution of the symptom, with no relapse. Incremental change looks more like: going to therapy for months or years, and experiencing only minor improvements in symptoms, and suffering multiple relapses in the proceeding years.

Unfortunately, there is substantial literature suggesting that most of therapy falls under the latter bucket. As psychologist Jonathan Shedler puts it, “Empirical research shows that “evidence-based” therapies are weak treatments. Their benefits are trivial, few patients get well, and even the trivial benefits do not last.” It’s difficult to measure the effectiveness of therapy as a whole, and it has been the subject of contentious debate for decades, but the overall sentiment after thousands of studies and meta-analyses is that therapy definitely helps, but it doesn’t help that much.4

Ecker and team believe that therapy could be helping much more. And in their view one of the major trends holding the therapy world back is the widespread focus on counteractive approaches (i.e. suppressing the symptom rather than targeting its root cause). In their book, they make the claim that all transformational change in therapy involves a process of rewriting unconscious emotional beliefs, even when this is not what the therapist is explicitly trying to do. They document dozens of case studies from across different therapy schools to demonstrate their point (emotion-focused therapy, psychoanalysis, internal family systems, and many others).

But are all psychological problems really about updating beliefs?

Is there an unconscious belief that underlies depression, and eating disorders, and alcoholism? What about perfectionism, compulsive behaviors, or aggression? The view of the authors of UTEB is that all psychological problems—aside from ones that have clear biological causes like genetics or hormonal deficiencies—are in some way based upon unconscious emotional learnings, and as a result they can be resolved with the right “disconfirming” experiences. Here are just a few examples from the book of how symptoms can be driven by unconscious beliefs, and resolved quickly once the belief is uprooted:

• Ted, 33, has a problem of pervasive underachievement, unable to hold a job or relationship for longer than a few months. Ted also had a difficult relationship with his father, who was constantly mean and disapproving of him as a child. In the course of a few sessions, the therapist helps Ted uncover the belief: “if I succeed in life, my father will never know how terrible of a job he did parenting me; I have to do poorly so that he feels remorse.” They then update this belief by helping Ted realize that, given what he knows about his father, he will never get the remorse he’s looking for. Ted’s life then turns around: he enters a long-term relationship, completes vocational training and gets a job.

• Brenda is an aspiring stage performer, but has stage fright that paralyzes her at her weekly rehearsal. During therapy, Brenda identifies the feeling of stage fright with a childhood memory of her being terrified as her drunk father was driving her family. This experience engendered a belief that Brenda was powerless and stuck. Together, Brenda and the therapist do a “reenactment” of that childhood experience, in which Brenda assertively asks her father to stop driving, and she gets help from other adults. Brenda effectively updates the mental model that she was powerless in that situation, and this relieves her stage fright.

• Raoul came to therapy with excessive and unpredictable bursts of anger. Over a few sessions the therapist helped identify that the anger is triggered by instances of a shared agreement being violated, even very minor violations. They then uncover that Raoul’s anger was connected to an experience years ago in which a business partner had betrayed him. This betrayal had resulted in Raoul forming a number of unconscious beliefs, including “Breaking agreements ruins lives. Anyone who breaks an agreement with me doesn’t care about ruining me and is my enemy. Without my anger, I would feel powerless and defenseless against being betrayed again like that, so I need my anger.”

Do these examples generalize to all cases of (non-biological) symptoms? I’m of two minds about this: on the one hand, any given psychological problem can be stretched and squeezed to fit the narrative of “symptom-producing belief” and “disconfirming evidence which updates that belief.” At the same time, this model really is powerful, and does explain the vast majority of human behavior, whether it’s in the context of formal therapy or more mundane psychological problems. And the dozens of case studies in the book show that by targeting the mental model specifically, lasting change can be achieved very quickly.

Ecker and team’s sell is that this framework makes therapists more effective, because as a therapist you have a clearer picture of what exactly you’re trying to do (identify and update emotional learnings), regardless of the specific technique you’re using to do it. It’s ultimately a playbook for transformational change, which is sorely needed, because as we’ve covered, much of therapy today is not transformational.5

My personal takeaway from reading the book has been to be treat my own problems differently: to always try to identify the specific belief underlying my maladaptive behaviors, rather than immediately looking for ways to correct them. Sometimes merely bringing a belief to conscious awareness reduces its power over you, because you immediately see how it’s not always true. For example, I recently had an argument with a close friend that left me feeling very unsettled, and I got immense relief from recognizing an implicit belief: “I need this person to believe that I’m right, because I don’t have enough confidence in my own judgement.” Simply recognizing this implicit belief helped me get a bit of distance from it, and reduced the emotional weight of arguments with my friend.

Time will tell whether Unlocking the Emotional Brain will accomplish the tall order of unifying all the schools of therapy. In the meantime, it has done the best job I’ve seen at articulating a basic tendency of human psychology: a lot of problems you can’t seem to solve are secretly solutions you’ve adopted to problems you don’t want to admit to having.6 What’s profound about this is that once you’ve identified the “underlying” problem, solving the original problem becomes much easier. The best part of Unlocking the Emotional Brain is that it encourages an attitude we desperately need more of in our culture at large: we can solve our psychological problems, and we can solve them much more quickly and decisively than we think.

Thanks to Susie for feedback on earlier drafts.

Let’s start with a basic premise: deflationary theories of “art” are no good. Some people claim that art is just “whatever anyone says is art.” To me this claim belongs in the same family of pseudo-insights as “truth is just whatever anyone feels like is the truth.” Yes, there is some subjectivity in both truth and art, but to deflate it entirely to “whatever anyone says”, and to claim that there is nothing more to say on the matter, is dishonest. If “art” and “not-art” were entirely made up and arbitrary distinctions—and by extension “good art” and “not-good art”—then we would never have had awards for art to begin with, we could never have the notion of “classics”, of reviews and ratings and critical acclaim, we would find it utterly impossible to converge on these things; galleries would never get up and running in the first place because it’s simply too subjective to figure out what to display. If “art” and “the goodness of art” was a totally arbitrary category, we would never talk about “the best art”; having awards for “the best art” would seem as absurd as awards for “the best color” or “the best childhood memory” or “the best dreams” or “the best morning routine.” All of these things—dreams, memories, morning routines—are things we talk about and care lots about, and yet we don’t have awards for them, because it is impossible to converge on a globally shared notion of “better” and “worse”—it is actually too subjective. That is not the case with art.

What people are really doing when they claim that “anything can be art” is synonymizing “art” with “beauty” or even “pleasure.” Art is just anything that looks nice or feels nice. I agree with Erik Hoel that this is overly reductive: the whole point of a word is to differentiate one thing from another, and if “beauty” is all that makes art, why not discard the term “art” entirely? A waterfall is beautiful, and so is the night sky, but neither of them are rightly considered “art.” In my view there was no art on this planet until humans (and perhaps some non-human ancestors) came along and created it. Art requires some degree of intentionality: it’s the result of someone trying to express something. (That “someone” need not necessarily be a biological human—but they do need to be an agent with the capacity to interact with and represent the world.)

Why does it matter whether a human (or some other agent) created a work of art? Isn’t the beauty and meaning of the art inherent to the work itself? In his essay critiquing AI art, Chiang gives a number of examples for why this is generally not true—we generally do care about the intentionality behind what we consume, rather than only caring about the pure information content. You would be more excited to read a postcard that your friend wrote to you than a postcard that someone else wrote for their own friend, even if the contents of the two postcards are roughly the same (e.g. “miss you and hope you’re doing well”). Similarly, we care about whether we’re watching Elvis Presley live, or just a guy who looks and sings almost exactly like Elvis Presley. We care not just about the raw sensory stimulus, but about what lies behind it (both in physical space, and also in time). We would pay more to see the Mona Lisa than a replica of it; we care more to see Taylor Swift in person rather than a Taylor Swift impersonator who has practiced how to look and behave exactly like her. It’s the whole causal history of interactions leading to the current moment that makes a given communication or work of art meaningful to us.

You might dismiss this as pure sentimentality. But what I would like to posit is that this is how meaning in general works: meaning is created by a process of interaction, rather than being inherent to the information content of the message. I will try to show that this is true for both senses of the word “meaning”: there’s semantic meaning (like when someone says “what does the word bequeathment mean?”), and there’s also existential meaning (like when someone says “that was a super meaningful experience for me”). Both kinds of meaning arise from interaction rather than being purely a subjective feeling or being inherent to the content.

The particular view of meaning I’d like to put forth has a pretty deep influence on the way you experience language and thought as a whole, and it also happens to be one of the theoretical foundations for the current generation of language models, so it’s worth explaining in detail. Of course, “the problem of meaning” is something philosophers have grappled with for many decades, and will be hard to completely settle in the course of a few paragraphs—but that is what I’m going to attempt to do, before getting back to talking about AI.

Meaning is interactive

Let’s talk about an example that will seem silly at first but will hopefully illustrate my point.1 Imagine a shepherd needs to keep track of his sheep—he has a bunch of sheep in a big pen, and every day he lets them out to roam freely in the field, and he needs to make sure all of them have returned at day’s end. To keep track of the number of sheep that have left his pen, he devises a simple method using a bucket and a bunch of pebbles. The bucket starts empty, and every time a sheep leaves the pen, the shepherd drops a pebble into the bucket. Every time any of the sheep re-enters the pen, the shepherd takes a pebble out of the bucket. In this way, the bucket of pebbles tells the shepherd exactly how many sheep are currently outside of the pen, so he knows at the end of the day if any of them are missing. Thus a notion of “meaning” is born: the meaning of the pebbles is the number of sheep who have left.

And now the question is: how did this meaning come about? Do the pebbles “reflect” the sheep in some physical sense? Do they have some magical counting property? No. It’s the shepherd’s actions that give the pebbles meaning. If the shepherd stops keeping count and throws away the pebbles, they no longer have any meaningful relationship to the sheep. If it just so happens that someone else walks by with a bucket of two pebbles, and at that moment there happen to be two sheep outside of the pen, we wouldn’t say that the second bucket also represents the number of sheep outside the pen—it only does so by coincidence.

The point is: meaning arises from intentional activity. It’s the shepherd’s ongoing commitment to keep the pebbles “in line” with the state of the world that gives them meaning. Meaning a process rather than a specific state at a specific time—or as David Chapman puts it, it’s not strictly subjective nor strictly objective, but interactive.

Now, the rather disorienting part of this is that you can apply this point to more than just pebbles in a bucket—it applies to language in general, and it even applies to our thoughts. When you write down on a piece of paper (or have the thought in your head), “get a carton of eggs and a gallon of milk tomorrow”, what gives that utterance its meaning is nowhere to be found in the shape of the symbols themselves. The symbols on their own are just a bunch of squiggles—there is nothing in the structure of the squiggles making up “egg” that corresponds to actual physical eggs. The symbols derive their meaning from the way they’re used in a specific cultural context (i.e. the English-speaking world). The meaning comes from the way that we, collectively, use those words.

We all have a very deeply held intuition that the abstract thoughts in our head are intrinsically meaningful and simply refer to things in the world, purely by virtue of their internal structure. But this is a mistaken intuition, and here’s one of the more obvious reasons why: it’s possible to be totally convinced that your thoughts/utterances are meaningful, even while they are completely devoid of meaning. This is one of the failure modes of specific kinds of philosophy, and is something anthropologist Dan Sperber talks about in his paper The Guru Effect. Sperber lists a number of examples of famous philosophers with devoted followers, who tended to make statements that were at best unclear, and at worst completely nonsensical. Consider, for example, this paragraph by postmodern philosopher Jacques Derrida:

“If différance is (and I also cross out the ‘is’) what makes possible the presentation of the being-present, it is never presented as such. It is never offered to the present. Or to anyone. Reserving itself, not exposing itself, in regular fashion it exceeds the order of truth at a certain precise point, but without dissimulating itself as something, as a mysterious being, in the occult of a nonknowledge or in a hole with indeterminable borders (for example, in a topology of castration).”

Do you understand what Derrida means here? Does he mean anything? I’m certain that he felt like he meant something by this statement, and maybe it does actually mean something; and yet it’s equally plausible to me that it has as much meaning as the babbling of an infant, as much semantic content as me saying “Gloobly flurp zibbit mox frandled squerking noobles plinky doo.”2 How do we figure out whether Derrida’s statement falls under the category of “meaningful” or “meaningless”? My point is that we can’t answer this question by parsing, in ever more detail, the specific words that he uttered; we answer it by stepping back and looking at how his words were used in the world, how they influenced his and his readers’ behavior. As Wittgenstein put it: meaning is use.

Now, everything I’ve said so far applies to semantic meaning, but what about the second kind of meaning—the existential kind? My sense is that interaction is also a requirement for this kind of meaning. When you say an experience was very “meaningful” for you, what that generally means is that it has a large number of causal connections to the way you see the world and your future actions. To say that an experience—or a work of art—is “meaningful,” is more than to merely say that it’s intense or pleasurable; it’s to say that the experience or artwork has tangible implications for your experience and actions. Again: meaning is use!

Back to generative AI

My point with elucidating all of this is that I agree with Chiang’s general stance on meaning. At first glance, when you chat with ChatGPT you might say: clearly everything it’s saying is meaningful. When it says “the capital of France is Paris,” it’s talking about actual things in the world and it has some understanding of what it’s talking about. It can reason and think in much the same way a human can.

But Chiang makes an interesting observation:

It is very easy to get ChatGPT to emit a series of words such as “I am happy to see you.” There are many things we don’t understand about how large language models work, but one thing we can be sure of is that ChatGPT is not happy to see you. A dog can communicate that it is happy to see you, and so can a prelinguistic child, even though both lack the capability to use words. ChatGPT feels nothing and desires nothing, and this lack of intention is why ChatGPT is not actually using language. What makes the words “I’m happy to see you” a linguistic utterance is not that the sequence of text tokens that it is made up of are well formed; what makes it a linguistic utterance is the intention to communicate something.

“ChatGPT is not using language” is definitely a provocative way to put it, but Chiang has a point. Language models are notorious for their tendency to hallucinate—like when GPT-3 said things like “Egypt was transported for the second time across the Golden Gate Bridge on October 13, 2017”—which might make us wonder whether the model really “means” what it says, or whether there’s some deep sense in which is “untethered” from reality and lacking genuine understanding of its utterances. You wonder if ChatGPT is, first and foremost, a bullshitter.

This would align with the notion of meaning we just described: if meaning is born of interaction with the world, language models evoke very little of it because they do very little interacting with the world. Yes, they go through a long training phase in which they ingest unfathomable amounts of text (and images, etc for multimodal models), and they do interact with humans through text boxes, but they rarely use language in the way that humans do. This could be one reason why they need so much more text than we do in order to be able to speak like us: whereas a baby can learn what a “cat” is by having an adult simply point to one and say, “that’s a cat”, there is no such real-world “pointing” available for an LLM. The LLM can only learn what a cat is by virtue of the way that the word “cat” appears next to other words. The LLM never learns how language is used in the world—only the internal structure of language itself.

Here’s a concrete, present-day example of a language model not really “meaning what it says”: if you ask ChatGPT or Claude to play a game of twenty questions, they will happily oblige, claiming that “I have an object in mind” that you are supposed to guess by asking it yes/no questions. The problem is, it’s impossible for the language model to actually “keep an object in mind” across a multi-turn conversation, without writing out that object into the chat itself, because current-generation LLMs are stateless: the only information that persists between messages is the text content of the chat itself. Of course, when you play the game, it sure seems like the LLM is keeping an object in mind, because it does keep all of its answers consistent with past responses. But the reality is probably more that the LLM “converges” on an object randomly over the course of the conversation.3

So: LLMs have a lot of difficulty with genuine meaning-making because they are not autonomous agents embedded in the world. But still, these models have gotten way farther than anyone could have reasonably expected, simply by looking at the internal relationships of words and images, rather than how any of these things are actually “used in the world.” This is why, despite all the differences between humans and generative AI that I’ve outlined—all the apparent limitations of language models—I have difficulty asserting that they are still just “stochastic parrots,” that they truly understand nothing of what they are talking about.

Instead, I think it’s better to think of “capacity for meaning” as a spectrum. A given person, or AI model, can have a higher or lower tendency to utter meaningful versus meaningless phrases, depending on their age, personality, and what they’re held accountable to. Infants babble nonsensically, and even teenagers and young adults are liable to parroting phrases whose meaning they don’t fully grasp until they’re much older. We see this contrast again and again in different kinds of people. Some politicians speak candidly and concretely, while other politicians can’t stop spewing verbal salad. The best business leaders speak in a way that genuinely motivates employees, while mediocre managers can’t help but put out bland corporate-speak. There are genuine spiritual leaders who provoke insight and discernment in their followers, and there are pseudo-enlightened gurus and intellectuals who utter fashionable nonsense. As humans, we are capable of not only being wrong, but being so confused in our statements that we are not even wrong. Generative AI, by virtue of being an unconscious,4 disembodied thinking machine, has even greater capacity for nonsense than us, because it has the least “grounding in reality.” So we should treat its utterances with a cautious distance, remembering that, just like the devout followers of fake spiritual gurus, an utterance or a work of art from AI can feel meaningful to us and still be completely meaningless.

Which brings us back to art. Just as these models have a high capacity for bullshit in language, they also have a high capacity for bullshit in art. But this is where I depart from Chiang: this doesn’t mean that they are incapable of participating in a meaningful act of communication or expression. It is precisely their use by humans (i.e. agents with intentions that are capable of representing the world) that imbues their creations with meaning.5

Art as it’s used, and the semantic apocalypse

If we put aside philosophical speculation about meaning, what is art, as a practical matter? Art is stuff we put in galleries for public consumption, or display in our bedrooms to make them feel like home; it’s stuff we use as our desktop and phone backgrounds; it’s stuff we gift our friends; it’s stuff we turn to when we feel lost or when we seek consolation. Art is also stuff we create in order to express ourselves; it’s an activity we engage in to build a sense of joy and mastery. It’s fine for AI to serve some of these functions, or to support us in fulfilling some of them. But I don’t want a world where AI art serves all of these functions, where it has replaced human-made art everywhere.

And yet, I have trouble imagining that ever happening. Despite the fact that today we have photography and digital art, people still paint things on actual physical canvases with actual physical paint. Photography and digital illustration never “automated” painting; likewise I don’t think it’s correct to think of AI as “automating” art. As Chiang points out, art is something that necessarily involves creative choices; the whole point is to take responsibility for as many of those choice as you feasibly can, rather than deferring all of them to an automation. In this view, generative AI becomes a tool just like cameras and digital illustration software.

Chiang claims that generative AI won’t become a tool for artists because the companies behind these models won’t tailor them towards people who want to put a lot of effort into the creative choices involved in making true art. This seems like a failure of imagination to me. Chiang’s own article describes the case of film director Bennett Miller, who exhibited AI-generated art at a gallery after generating over a hundred thousand images and whittling them down to just twenty. But he asserts that his won’t happen generally—that AI companies won’t make models that allow you to “enter tens of thousands of words into a text box to enable extremely fine-grained control over the image you’re producing.” And yet, someone commented on Chiang’s piece claiming that they’re already doing exactly that. As much as there are market incentives that might push these tools toward convenience over control, there will be others who seek out that control and build it for themselves.

My sense is that the most vocal critics of AI art are worried about something like a semantic apocalypse. That by virtue of AI-generated content being untethered from actual physical embodiment and interaction with the world, our culture will devolve into a shadow of itself, full of replicas and lookalikes, devoid of real meaning. The concern is that this transformation will happen slowly—too imperceptible to notice day by day, but totally unmistakable in retrospect. Just as LLMs can collapse into meaninglessness when recursively trained on their own outputs, the fear is that our culture will stagnate and ultimately devolve, and will look much like the Facebook feeds that are increasingly full of eery, AI-generated crap.

A future like this seems deeply disturbing—it’s a Brave New World saturated by superstimuli. Like male beetles that mate with beer bottles because the bottle looks more like female beetles than actual living females, we might become enamored by the artificial and disengaged from our actual lives. As Vivid Void puts it, we might build companions that “start as an AI friend to share jokes with and end as an LLM trained on your ex-wife who left you, or your deceased father, or the best friend who cancer took too soon.” We might lose our capacity to come face to face with grief, and loneliness, and all the other ills of real life, numbing ourselves instead with a pale substitute for real connection.

But humans are not beetles. I believe in the inextinguishable wisdom of the human spirit, in our desire for truth and genuine meaning. To the extent that AI is incapable of meaning-making, the most discerning among us will be capable of recognizing it as such. As humans, we can tell that superstimuli are actually understimulating—that’s why we decry them, because we can tell that something is missing. It’s why people choose to step away from their phones and social media, why “touching grass” is a meme, why we create minimalist computers that prioritize the user’s wellbeing rather than exploiting their compulsions. We won’t prevent a semantic apocalypse by insisting on particular definitions of words; we will do it by trying as hard as possible to create genuine meaning. The wisest among us will continue to do this, and they will inspire others to follow suit. It will always be true that individual humans can be fooled by a false idol, and sometimes even entire groups can be lured in by the simulacrum of meaning. But humanity as a whole will never be.

Thanks to Arkady and Ben for comments on earlier drafts.

Doing philosophy helps you get better at constructing and deconstructing arguments; surprisingly, this does not always make you better at figuring out the truth.

If you’re really good at constructing arguments, you will be better at convincing yourself of any particular belief you’d like to have, regardless of whether it’s accurate. You get better at answering the question: “what are the ways in which this might be true?” and “what are the ways in which this might be false?” rather than the question “is this true or false?”

Constructing and deconstructing arguments is one part of being in touch with the truth, but the other, equally crucial part is being able to deploy attention and emotion skillfully. Being attuned to your emotions helps you better understand your motivations around a particular belief, which is crucial for assessing its accuracy. Traditional philosophy doesn’t help you with this because it doesn’t encourage you to introspect.1

When I originally discovered philosophy, what I found particularly appealing about it is that it seemed like the king of intellectual pursuits, because it was asking the deepest questions. What is the fundamental nature of reality? Where does ethics come from, does it exist in any objective sense? But just because you are asking the questions—and constructing and deconstructing arguments about them—does not mean you are getting closer to answering them. For both of the above questions, we have spent thousands of years arguing about them, and haven’t made much progress.2

These days I think the “ladder of abstraction” is very important when thinking about philosophical discussions:

The idea is that the things at the bottom of the ladder are very “concrete”, i.e. they are things you can fairly easily point to (like the chair you’re sitting on), whereas the things at the top of the ladder are much more “abstract”, they are much broader (e.g. “wealth” or “justice”). I believe this concept originally came from S.I. Hayakawa’s Language in Thought and Action:

The point of the ladder is that there are basic facts that are very easy for us to agree upon, and then there are facts that are higher up that are very hard for us to agree upon. The lower levels of the ladder give you a bunch of “consensus for free.” No matter your political beliefs or philosophical inclinations, you will agree that in front of you is a cow. No matter your worldview, you will agree about whether it’s daytime or nighttime.3

Here are two conversations about the color red, one of which moves down the ladder of abstraction and the other moves up (taken from this article):

Case 1: Moving up the ladder of abstraction (less helpful):

“What is meant by the word red?”

“It’s a color.”

“What’s a color?”

“Why, it’s a quality things have.”

“What’s a quality?”

Case 2: Moving down the ladder of abstraction (more helpful):

“What is meant by the word red?”

“Well, the next time you see some cars stopped at an intersection, look at the traffic light facing them. Also, you might go to the fire department and see how their trucks are painted.”

The higher up the ladder you go, the looser your grip on reality gets. Philosophy exists purely at that highest level of abstraction. Rather than speaking of, for example, how much water your plants need to be fed to stay alive, you talk about whether, for example, propositional knowledge captures everything there is to say about reality, or whether it’s possible that abstract patterns in non-sentient matter could generate conscious experiences. It’s very high up the ladder. Far away from action, far away from specificity, far away from things you can see and touch and taste. This doesn’t mean that it is totally devoid of substance: just that it is harder than in any other field to know whether you’re studying something of substance or not.

Here’s a specific way that this played out in my own life. I spent a lot of time being obsessed with the ideas of Karl Popper and David Deutsch. I was totally enamored by the universality of explanations, by conjecture and criticism as the fundamental process driving science and society, by a theory of knowledge and of biology centered on error-correction. Now I look at all these ideas and I think: meh, there isn’t that much there, they are not that deep because they don’t map to the real world as much as I thought they did.4 But a few years ago I was spending many dozens of hours arguing, futilely, with friends about how these are really such deep and profound and important insights!! And I could never quite convince anyone who wasn’t already really into philosophy, and really into Deutsch and Popper specifically, why they mattered.

Because of the nature of philosophy, you have to be really good at introspecting in order to do it well. You have to have a deep understanding of your own thought processes and emotions. I don’t just mean more theories about your emotions; I mean actually knowing yourself, knowing the ways you tend to lie to yourself, knowing the things that trigger you, being able to experience those triggers without your awareness totally collapsing. Not knowing yourself, in the sense I’ve described, will do more damage to your ability to think clearly than any amount of philosophical inquiry can compensate for.

Does this mean philosophy is always a waste of time? No. First of all, there is a richness to exploring these questions, assuming they are the kind of question that tickles you. When engaged with skillfully, philosophy can make you more appreciative of the beauty and mystery of existence. Also, there are practical benefits to getting better at constructing and deconstructing arguments—you become a slightly better thinker, writer, scientist, et cetera. You become harder to fool by others, even though in some sense you become better at fooling yourself.

Beyond that, I think the benefits of philosophy are overstated. In particular, I think people who cling very strongly to the idea that you need to do philosophy to understand the world and live well are wrong. “Philosophical beliefs” are largely irrelevant to our existence in the world; they are irrelevant to almost everything we do, except for very specific edge cases, like being a quantum physicist or being a neuroscientist trying to understand consciousness. If you’re trying to do those things, having a strong background in philosophy seems very helpful, otherwise it largely isn’t.

Now. Even with all those caveats mentioned, I have to admit that philosophy is something that continues to draw me in, because I’ve always had a soft spot for the deepest questions. In a recent tweet thread about the nature of math, mathematician David Bessis said that “mathematics and philosophy are the only two human activities completely of the mind,”5 and, well, I’m a sucker for activities that are completely of the mind. I love it when I can find a small number of ideas that cover a lot of ground—they make me feel powerful, they give me some sense of control in a life that is otherwise full of overwhelming uncertainty. It’s just that, unlike a few years ago, I am much more honest with myself about these things—about the actual reasons I’m doing philosophy, and the actual impact it has on my life (which is often negligible). When I read philosophy now, I approach it less in the spirit of Finding The Ultimate Answer, and more in the spirit of “let’s read these fun little fairy tales and maybe they’ll give me a sense of clarity about something.”

My point about philosophy is that there is nothing necessary about it. Of course it’s good to look at the world with curiosity, and to think critically, but I think you can do all of that without philosophy. It’s very easy in philosophy to feel like you’re doing something important by talking and thinking a lot about deep questions — and so it will naturally draw people who enjoy thinking and talking, because it tells them that the thing you like to do most in the world is also cosmically significant. And in a very loose sense it is. But it is not uniquely so.

Thanks Sid for discussion and comments.

Appendix

A few other thoughts/nuances I couldn’t fit into this essay nicely:

• In the process of writing this post I realized I’m talking about a rather specific kind of philosophy, which some might call the analytic tradition, and which I would describe as “trying to talk very rigorously about deep meta-questions that are outside the purview of ordinary sciences and humanities.” This doesn’t capture everything that could conceivably fall under the umbrella of “philosophy” (e.g. in my view existentialism and phenomenology don’t fall under it). But I do feel like this specific kind of philosophy is overrepresented among people who talk about it on the internet, and so it’s the definition I stuck with.

• The astute reader would point out that the “ladder of abstraction” is itself an abstraction. And yes, it is. Anytime we use language we are employing abstractions. I don’t think this matters. The point is that there is such a thing as “more concrete” language and “less concrete” language and that it’s much easier to get lost in the sauce when you’re engaging in the latter.

  • Also, someone might point out that items on the lower level of abstraction are not actually easy for us to agree on; e.g. at what exact point is it no longer “daytime” and now “nighttime”? Different people can disagree about this, just as different people can disagree about whether a hot dog is a sandwich. In my view this results from the fact that any category we impose on the world is necessarily blurry at the edges.6 But there is still a crucial difference about the kinds of disagreements we might have at the lower vs higher levels of abstraction. At the lowest levels of abstraction it’s very obvious to the participants that the debate is merely a semantic one, i.e. it’s about arbitrary word definitions. What happens at the highest levels, though, is that it’s harder to tell whether your disagreement is merely about arbitrary definitions or whether it’s more substantive.

• There’s a specific class of philosopher who, in response to this post, would say: “well, your argument about philosophy being a waste of time is actually a philosophical argument; thus it refutes itself.” I used to be convinced by this argument and I no longer am, but it’s hard to articulate exactly why. Roughly, it’s something like this: if you take this view (that you can only refute philosophy with more philosophy), then philosophy becomes literally inescapable. In this view of the world you are constantly doing philosophy whether you realize it or not; and in the extreme it also means that babies are doing philosophy and so are monkeys and wasps. Another tenet of this worldview is that “if you believe X, that means you necessarily believe all the imaginable logical implications of X.” I think this is based on a fundamental misunderstanding of the nature of human belief. So, I don’t find it convincing. You don’t have to refute philosophy by doing more philosophy; you can just refute it by putting it aside and living life.

Competition and cooperation are both ubiquitous in biology. I don’t just mean at the level of organisms – we’re all familiar with the ways that organisms compete with each other (the “Darwinian struggle for survival” where only the fittest survive), and also with the ways they cooperate (ants form colonies, birds form flocks, fish form schools). What was surprising to me is that competition and cooperation occur within individual organisms as well.

Cooperation within the organism is pretty easy to see. The whole body can be viewed as a chaotic jumble of overlapping, cooperative processes. The cells of your nervous system are constantly in communication, computing transformations of sensory input and inner thought that are still far beyond our understanding. The cells of a developing embryo morph from generalized stem cells into specialized tissues depending on where exactly in the body they are. And your immune system coordinates to identify the shape of foreign pathogens and then unleash an army of fighters seeking out that pathogen, to protect your body against infection.

And yet, your body is also a battleground for internal competition. In most regions of the brain, half or more of the neurons that are born die out, suffocated by the competition for limited resources. In the embryo, between days 6 and 7 of development, a full 35% of cells are quickly eliminated, leaving only the most vigorous cells to form the body. And in the thymus—one of the core organs of the immune system—younger stem cells are constantly outcompeting and destroying older stem cells, in a process that’s crucial for the prevention of cancer.

Since the publication of Darwin’s Origin of Species in 1859, which demonstrated that the driving force of evolution is competition for resources among organisms with varying genes, biologists have wondered whether a similar kind of competition takes place at the level of cells, tissues, and organs. In the past few decades, a wealth of evidence has uncovered that it does: we’ve identified competitive processes in tissues across the body, and in organisms across the animal kingdom.1

Just as humans can determine each other’s “fitness” by evaluating their physical characteristics (to make decisions, say, about who to mate and who to fight), cells use signaling mechanisms to determine the fitness of their neighbors. One of these mechanisms is a protein called Flower, which cells display on their surface, and different variants of the protein (the “losing” variant and the “winning” variant) communicate the cell’s internal state to other cells. When a “winning” cell is next to a “losing” cell, the winner actively triggers the self-destruction of the losing cell, through apoptosis, or programmed cell death. This process underlies a number of crucial developmental and homeostatic functions in the body—like choosing the most vigorous cells for the development of skin tissue, culling stem cells that don’t differentiate correctly, and eliminating cells that have incorrect numbers of chromosomes. Crucially, the signaling proteins are necessary for this elimination to take place: if you remove the signaling protein, the “less fit” cells are not eliminated—they persist, despite being less functional than the fit cells. In other words, it’s not just a blind competition for resources: there are precise mechanisms that determine the winners and losers and that target the losers for destruction. This signaling process seems to be conserved throughout the animal kingdom, whether you’re looking at flies, mice, or humans.

We might ask whether these processes are really “competition” or just more examples of “cooperation,” and it depends partly on the perspective you take. On one level, cells really are competing with each other, because as individuals they have objectives that are in direct opposition.2 To see this, consider two neurons both attempting to form a connection with a muscle fiber. Usually what happens is that one neuron “wins” and covers the muscle fiber while the other neuron “retreats.” But how do we know the neurons are “competing”, versus collaborating as a pair to “choose” one of them to take over? Well, it turns out that if you injure the neuron that has won, the losing neuron—which had previously retreated—actually comes back and takes its place. The losing neuron was not “retreating on its own accord”—it was genuinely “competing” with the other neuron and capable of taking its place as soon as it had the chance to.

At the same time, individual neurons, and all other cells in the body, can be viewed as smaller agents within a larger system, and that system is itself an agent (you). From the perspective of that agent, all of this competition is profoundly helpful, it is keeping the system healthy and alive. The competition follows specific rules and has specific purposes, making it a kind of cooperation. Despite the fact that there are winners and losers (cells that survive, and cells that die), the system as a whole is better off for it.

Of course, competition is not always productive for the collective. These same competitive dynamics that are crucial for healthy functioning can be hijacked by cancer cells. Cancer cells have been shown to, among other things, disrupt the ordinary signaling that determines winners and losers in cell competition. Understanding the dynamics of these competitive processes could help lead to new approaches in cancer therapy. We’ve found, for example, that competition within an individual tumor (between different classes of cells) can stop the tumor’s growth; and that often, the growth of a tumor is impeded by other hyper-competitive (but not cancerous) cells that surround and smother it. More research still needs to be done, but among the potential therapeutic approaches are ideas to boost the fitness of healthy cells around developing tumors, or to introduce “more fit” cells through transplantation.

Zooming out beyond the dynamics of individual cells, some biologists speculate that competition between entire organs and limbs also plays a functional role, specifically in the development of high-level structure. Experiments on butterflies have shown that if you remove the butterfly’s hindwings in early development, its forewings grow to be substantially larger, suggesting that the particular size of the wings is determined by competition for limited resources during growth. Similar results have been found in beetles and fruit flies, and there is (as yet contentious) speculation that competition regulates the size of mammalian embryos. And a recent computational study suggests that evolution actually opts to employ a finite resource for cell processes—even when it has the option of employing an infinite resource for the same processes—because the finiteness of the resource offers a coordination mechanism for the whole body. The fact that a resource is finite enables it to form a “global scratchpad” for the body: relative levels of the molecule give you information that you wouldn’t have if the molecule was abundant everywhere.

There is still much to discover about both the detailed mechanics of competition between cells, and the higher-level functions that this competition might serve. But what is abundantly clear is that competition, even within the body, is not always a bad thing: it can just as often be constructive as it is destructive. This is the case in every multi-agent system, be it an organism or a society—competition is sometimes beneficial for the collective (e.g. sports competitions pushing the boundaries of athletic achievement) and sometimes detrimental (e.g. wars leading to irrevocable death and destruction). It’s the finely-tuned interplay of competition and cooperation, across scales from cells to bodies, that makes life work.

Thanks to Susie and James for feedback on earlier drafts.

Machine learning is already being used extensively in science, helping us model everything from the weather to the folding of proteins to the interactions of electrons. Neural networks help us not only to make more accurate and efficient predictions of physical systems, but they’ve also shown they can predict emergent phenomena they had never seen before. In an especially exciting result this month, a neural network that was trained on simulations of atoms accurately predicted behaviors like the formation of crystal lattices and water molecules exchanging protons, even though it had never seen such behavior in training. A friend explained it like this: imagine you train a neural network on images of stars, without ever showing it a black hole or supernova. Then, you prompt the model to show you “a dense star”, and then “a very dense star”, and then “a very very dense star” and so on, and suddenly the model starts giving you pictures of black holes and supernovas, even though it had never seen one in its training data. This is what we’re doing but at the atomic scale: the model is learning something fundamental about how the atoms work, rather than just pattern-matching what it has seen in training.

This is all exciting, but one problem with applying machine learning to science is that collecting the required data1 is expensive, and training a model is also expensive, and scientific institutions—well, they don’t tend to have the kind of money that big tech companies have. Add to that the fact that each particular physical system has a slightly different set of equations that describe it, with different parameters and initial conditions2, which requires a different neural network to model, and you have a very expensive problem. This is where the prospect of “foundation models” in science come in: maybe we can pre-train one large model on a bunch of different physical systems, and have it get better at simulating all of them, and then have individual research groups “fine-tune” the model to the specific system they’re interested in. This is how ChatGPT works: first it’s pre-trained on all the text on the internet, tasked with learning how to autocomplete text, and then it’s fine-tuned with human feedback to be a useful chat assistant—but it can also be fine-tuned for other tasks like giving medical diagnoses or generating SQL queries.

Multiple groups of researchers are now building foundation models for fields like climate and fluid mechanics, which learn to simulate systems across a number of conditions and spatiotemporal scales, and end up performing better than state-of-the-art models that are trained on more specific conditions. The team at PolymathicAI—a consortium of researchers on a mission to build foundation models in science—recently built a multimodal model of galaxies, showing that combining data from images of galaxies and their spectral distributions (i.e. the relative amount of radiation they emit at each frequency) allows us to accurately predict other properties like mass, age, and star formation rate better than existing models. And just like with language models, the accuracy of the scientific foundation models improves with scale. As they get larger they become more accurate, and they also become more general, requiring less “prompting” to accurately model physical systems they haven’t seen before. In a particularly strange twist, even a model that was pre-trained just on YouTube videos performed better at simulating fluid mechanics than a model that didn’t get any pre-training and learned the task from scratch.3

What could these models be learning that apply across different modalities and physical systems, and could even extend to cat videos? Just like in language, the idea is that there are a number of shared properties across science—symmetries, conservation laws, and even basic concepts like causality—that foundation models in science can learn. Professor Miles Cranmer of Cambridge makes the following analogy: when we pre-train the model on a variety of systems we’re giving it the equivalent of a high school science education and then asking it to solve graduate-level physics problems. But if we just take a model without any pre-training and immediately try to teach it a specific task, it’s like trying to teach a toddler to predict the fine-grained properties of black holes before it has even learned how to speak.

Now, the holy grail of science is not just more accurate predictions, but new theories: explanations of the things we observe that give us an intuitive understanding, which in turn can guide further research and technology. Here, too, machine learning can help: we are starting to learn how to take the neural networks we use in science, and distill them into human-interpretable equations. This is a method called symbolic distillation, where we effectively “factorize” the neural network into a bunch of smaller components, and then extract an equation from each of the parts of the network using symbolic regression. Scientists have used this approach to discover physical laws: a neural network rediscovered Newton’s law of gravity given the trajectories of planets orbiting a solar system, without knowing the actual mass of the planets or any physical constants. And it’s not just discoveries of laws we already know: we’ve used this same approach to discover new laws that govern the scaling of galaxies and the dynamics of cloud cover.

The primary promise of foundation models is to democratize the use of machine learning in science, by giving more people and institutions the ability to fine-tune powerful state-of-the-art models to their specific applications.4 In the long run though, they could give us not just more cost-effective and accurate predictions, but new insight. We can build large general-purpose models that “understand” a variety of physical systems from climate to astrophysics to molecular biology, and couple that with better methods for interpreting neural networks and distilling them into human-interpretable equations. What if, by training large models on physics, chemistry, and biology, we are bringing our models a step closer to “base reality,” giving them a new kind of knowledge of the world—knowledge that isn’t constrained by our perceptual faculties (as in image/video) or by our conceptual priors (as in language)—knowledge that even we don’t have yet? If there’s anything we’ve seen from the last few years, it’s that we should expect to be surprised by the capabilities of these models develop as we give them more compute and more data. It could be, as Cranmer puts it in a lecture, that the next great scientific theory is hiding somewhere inside a neural network.

Thanks to Suzanne and Andrew for feedback on drafts, and cover photo credit to Yves Pommier.

I. The Mystery, or, How an Egg Turns into You

What are the biggest mysteries in science? When we ask this question we tend to talk about big picture topics like dark matter, consciousness, aliens, and parallel universes. We don’t talk as much about something that is more commonplace yet equally astonishing: how does the human body construct itself out of a single cell, like when a single fertilized egg cell develops into an embryo and ultimately into a fully-fledged adult?

We don’t tend to ask this question because we’re used to it—babies are born, acorns turn into trees, and eggs hatch into chickens every day. But there is indeed something perplexing about it. Think about what your cells have to do in the process of constructing your body: they have to coordinate their positions to follow a detailed architecture of bones, skin, muscles, and organs; they have to construct and wire together the hundred billion neurons of your brain; each cell has to decide what kind of cell to specialize into, and how much to duplicate to ensure all the proportions of your body parts are correct. How do so many individual units cooperate to self-assemble into a large, functional whole?

Well, let’s first consider a very different kind of functional whole: the computers, cars, and other machines that make up our built environment. These things are also complex functional wholes that are made up of countless tiny parts. But they’re “dead” in the sense that they don’t construct and heal themselves. The only thing that enables all the tiny parts to work together is the fact that we designed and put together all the parts ourselves, meticulously executing a detailed plan. The trouble with biology is that you don’t get this kind of top-down planning: when an egg cell proliferates into a full body, there is no “central command center” that has access to the entire developing body and can dictate what each part does. There is no brain around to tell the body how to construct the brain itself.

In the absence of top-down design, our conventional picture of biology is that everything happens in a bottom-up manner: molecular mechanisms dictate the functions of cells, which dictate the functions of your organs and which ultimately control your body. What is the thing at the very bottom of this hierarchy—the foundation for everything else in life? The genome. Genes are considered the fundamental code of life, so when it comes to figuring out questions of how the body develops, or how to cure diseases or change specific biological traits, we tend to look there. We spend much of our research efforts doing things like studying the gene that triggers the construction of eyes, or trying to identify the genetic underpinnings of Alzheimer’s and cancer, or studying how mutations in a specific gene alters a fruit fly’s circadian rhythm Ever since we finished sequencing the human genome back in 2003, we’ve been inundated by genomic data that we don’t quite know what to do with, but the emphasis is primarily on how genes and chemical pathways determine the high-level structure of the organism.1

That is, until Michael Levin (and many others) entered the scene. They came in and said: genes are great, and they do contain much of the necessary information for building our bodies. But they don’t contain all of it, and they are not always a useful level of abstraction of understanding how the body develops, and consequently they are not always the best way to intervene with biology (e.g. to regenerate damaged organs, or to cure diseases like cancer). If you’ve ever done any programming, you know that there are many levels of abstraction—higher-level and lower-level programming languages, higher-level and lower-level API’s—at which you can try to understand or manipulate the software that runs in your computer. Levin’s point is that genes are like machine code, and modern-day programmers never think about machine code—they think about higher-level software constructs like objects, modules, and applications. The bold claim embedded in his work—the real revolution here—is that higher levels of abstraction and control meaningfully exist in biology. And one of the ways in which this higher level of abstraction manifests is in something called the bioelectric network of the organism.2

II. Bioelectricity, or, The Worm That Splits in Two

We usually think of neurons as the only cells in our body that produce intelligent behavior by communicating in large networks. Neurons are constantly communicating with each other in the form of electrical patterns on their membrane and neurotransmitters, which are chemicals that transfer messages between cells. But it turns out that cells throughout the body have the exact same building blocks for such communication. They do the same communication, but slower. Levin and company call this the bioelectric network, as distinguished from a neural network.

In the past few decades we’ve discovered all the ways in which bioelectric networks distributed through the body do the same kinds of things that brains do: store memories, solve problems, and guide development. To get a sense of the bioelectric network in action, we have to talk about a mind-blowing creature called the planarian. This little critter (about 2cm in length) is a developmental “genius” of sorts: it doesn’t age, it doesn’t get cancer, and it is extremely regenerative, capable of regenerating any part of its body that gets cut off, even if it’s cut up into more than 250 pieces.

The question we asked earlier—how an egg cell expands into a full body—applies equally to how the planarian regenerates its entire body from just a tiny piece of tissue. (We can think of development itself as a kind of regeneration event.) While there’s still much to understand about the planarian’s profound regenerative qualities, Levin and others have shown that the bioelectric network of the worm plays a crucial part.3

Imagine taking one of these worms and splitting it into two. You now have two half-worms, and each of those half-worms is tasked with rebuilding the rest of its body. There’s a crucial decision here that the cells have to make: what part of the body do we already have, and what part do we need to build? One of the half-worms needs to produce a tail, and the other half-worm needs to produce a head. But the cells are at the very middle of the body, extremely far (from a cell’s perspective) from both the head and the tail. How do the cells have any idea what they should generate?

The answer, at least in part, is that all along the body the cells of the worm have a gradient of “resting membrane potentials”, which is effectively a stable electrical state. The cells keep track of their “position” in the body in this way, and experiments have demonstrated that the cell’s electrical state relative to the rest of the body is what determines whether it will proliferate into a head or a tail.4

Why does all this matter? Because once we understand how cells coordinate to decide what parts of the body to build, we can begin to intervene with that development to create new body structures. And they’ve done exactly that: Levin’s team was able to induce the worm to generate two heads instead of one head, by putting it into a solution of drugs that blocked specific ion channels (which in turn altered the electrical state of the cells). They’ve also induced the worm to generate no heads at all, or to generate the head of a different worm species. All of these are living, functional worms, just with a very different body structure.

Keep in mind a crucial point: in all these experiments, the genes of the worms are never edited. You get a wildly different functional worm with the same genes. And what’s even wilder is that some of these changes are enduring: without any further drugs or modifications, the two-headed worm produces offspring that are also two-headed, indefinitely.5 Think about what this means: we’ve achieved a permanent change in the structure of the worm, without changing its genes. We have transcended the genetic code and are instead learning to crack the bioelectric code of the body.

The worm is just one example: Levin’s lab and others have already demonstrated an astonishing level of control over development by modulating bioelectric networks. They’ve done things like getting frogs to develop extra limbs, and getting them to develop an eye in their gut, or an eye in their tail that they can actually see out of. The end goal that Levin dreams of is an “anatomical compiler” – a program which takes as input a specification for an arbitrary organ or body plan, and outputs the specific set of chemical and electrical signals needed to generate that organ. Imagine 3-d printing entire synthetic organs and organisms, except instead of having to specify all the micro-level details, you can just give a high-level description like “an extra eye at the tail.” This is Dall-E but for biology. And in the very long run, it could be the answer to virtually all of biomedicine, including traumatic injury, birth defects, degenerative disease, cancer, and aging.6

III. Fractal Intelligence, or, Fractal Creativity

So on a practical level, the impact of Levin’s work is a shift away from genes as the only determinant of structure, shifting instead towards the bioelectric network. But there’s a broader thesis here, which is recognizing that the terms “intelligence” and “cognition” apply to much more of biology than we tend to think. The very process of development has an intelligence of its own: for example, if you take a tadpole (the precursor to a frog), and manually scramble its facial organs, those facial organs will relocate back to the correct place as the tadpole matures.

The developing body of the tadpole is actively moving towards a “goal” state: it’s not a hard-coded system that blindly follows a predetermined set of steps encoded in the genes. The manual scrambling the experimenters created (called “picasso frogs”) is a situation that would not have occurred in the evolutionary environment, so it can’t be something it was specifically selected for.

There are many more examples of such adaptability in biological systems, found at the level of individual cells and groups of cells. Levin defines “intelligence” as the capacity to achieve the same goal via different means, and over the years he and others have documented case after case of such adaptability. If a developing embryo is surgically cut into two, it develops into two separate, healthy twins rather than two half-bodies; the embryos restructure themselves on-the-fly in response to external perturbations. If a newt salamander’s cells are artificially enlarged, the tubules in its kidneys still develop to be the same objective size, by simply using fewer cells per tube. It’s like the cells are cooperating to produce a fixed tube size no matter how large the individual cells are. As the individual cells are made to be bigger and bigger, the salamander even develops tubules out of a single cell that turns in on itself:

So we have all these examples of fractal intelligence, at layers up and down the biology stack. But the profound flip-side of intelligence is creativity: biology is capable of not just recovering the same functionality when perturbed, but of adopting entirely new kinds of functionality when given the right set of prompts. Levin’s team has taken skin cells from an embryonic frog, and given them certain signals to create “biobots” that move on their own and even self-replicate. (Again: no genetic modifications, just taking ordinary stem cells and giving them drugs.) More recently they’ve taken cells from adult human lung tissue, and used them to construct moving biobots that can heal damaged neurons. There’s a whole world of latent creativity to be discovered at all levels of biology, and the potential applications are endless. Imagine creating little biobots that can attack cancerous cells, or clean up toxins in the environment, or heal degenerated nervous tissue.

The bigger, mind-blowing perspective shift in Levin’s work is to rethink what we think counts as an “agent”, with “goals” that it pursues. Is a cell an agent? An embryo? What about a liver? What about your immune system? Levin hypothesizes that way before evolution discovered agency and intelligent information-processing in brains, it had already discovered it in lower-level systems—in morphogenesis (the process by which an organism develops its high-level structure), in bacterial colonies, and even in networks of genes. These are systems that we have trouble viewing as autonomous entities capable of forming memories or having goals. But what I think is truly special about Levin’s work is that he doesn’t just advocate a shift in perspective from an armchair: his team has been putting in decades of experimental work demonstrating the validity and promise of these ideas. His focus has been not just on philosophizing about what is intelligence and agency, but instead asking: what definitions, what frameworks, will lead to the most fruitful lines of empirical research and potential applications? Once we view cells and groups of cells as having an innate intelligence, we can leverage that intelligence to our own ends.

It’s not just biomedicine that stands to gain from this perspective shift: if we think of our brains, our organs, and our cells as all having the same basic building blocks for cognition, then we can share tools and ideas from across fields. The notion of “cognitive science” expands beyond just the study of neurons in the brain, to any cell type that coordinates together, or really any collective at all, including groups of humans. Specific parallels have already been studied, like studying cancer as a “dissociative identity disorder” of cell groups, or the finding that ant colonies succumb to “visual illusions” of the same kind that brains do. Levin says that all intelligence is collective intelligence: all of these “diverse intelligences” are ultimately made up of parts, a large number of subunits that have their own competencies and lower-level intelligence, combining to produce something greater than the whole. We tend to think of ourselves as individuals, as an indivisible unit, but the same “collective intelligence” label applies to each one of us: you are ultimately a collection of a hundred billion neurons (and trillions of other cells) cooperating together, each cell having its own competencies and subgoals that sometimes even conflict. It's strange to think of ourselves in this way, our brains being on the same spectrum as a colony of ants or a flock of geese, but once you look into the details of how we function it's hard to see it any other way. Just as the human world is a society of selves, your body is a society of cells. The connection might be more than mere metaphor.

Credit to Suzanne for feedback on earlier drafts and this post for originally introducing me to Levin’s work.

Even with all the recent advances in technology—cars that can drive themselves and chatbots that are ~indistinguishable from humans—I still think the brain is the coolest thing ever.

I also think it’s good to write down your open questions. The questions you ask are a window into your mind: they contain an endless matrix of preconceptions and assumptions; they reflect what you think is interesting and valuable. They form a time capsule that your future self can look back on with nostalgia (and embarrassment).

Every now and then the world is changed by a question no one had dared to ask before. Newton asked why an apple falls perpendicularly to the ground rather than sideways or upwards, and out came his universal theory of gravitation.1 Einstein asked what it would be like to ride a beam of light, and revolutionized our understanding of space and time, which later helped us build satellites and land on the moon. The next great scientific revolution is hidden under a question we haven’t bothered to ask yet.

Some people say there are no bad questions. I think there are bad questions, but it’s only by asking the bad questions that you can start to figure out what the good questions are. And it’s only by writing your questions down, by speculating and discussing them with others, that you’re forced to think more deeply about them. You have to be willing to entertain questions that seem silly or obvious to discover something new.

In the spirit of indulging our curiosity—of asking questions both good and bad—here are some questions I have about the brain, paired with commentary and speculation.

Q: How come even when we are not doing anything that requires thought/mental effort, our mind goes on continuously thinking about things, wandering in random directions, instead of just sitting silent? How come we have to work to get our mind to quiet down?

I’m so used to the concept of a wandering mind that I never bothered to ask this question until I saw it in a book. One proposed answer is that mind-wandering helps us with memory consolidation—e.g. we replay recent experiences in order to “sharpen” them and differentiate them from other memories, helping to make each individual memory more discrete.

Another way I tend to think about this is that mind-wandering is our subconscious’s way of “bubbling up” memories/thoughts it deems important to our conscious awareness. And the subconscious tends to do this more often when we are actively suppressing some part of ourselves, e.g. we are resisting an emotion we don’t want to feel. The more tension we hold, the harder it is to keep our mind still.

Q: Among those of us who read a lot of novels, how come we never confuse events that actually happened in our real life versus events we “experienced” by reading them in a vivid story? As entertainment gets more immersive (e.g. mixed reality) are we likely to make this kind of mistake more often?

When I posed this question to a friend, he pointed out that some of us do confuse things that happened to us with things we heard about, especially for distant memories. But I still think it’s significant that this confusion doesn’t happen more often. Here’s one answer for why: when we directly experience something, it’s a much richer experience than having a story recounted to us, and this makes it fairly easy to distinguish things that actually happened from things we read about.

Contrary to this view though, there was an interesting case study of a patient who, after a brain injury, could still remember all his memories, but no longer felt like they were his memories. They just felt like stories that he had read or heard about from someone else. As his brain gradually recovered, individual memories would start to feel like “his own” again, one by one. This suggests that it might not just be the content of the memory (e.g. its vividness) that tells us whether it was our own, but some additional “I-was-there-at-the-time” property that our brain attaches to it, which can be selectively removed.

Q: How come we are never surprised by the non-sequiturs in our dreams (and in our trains of thought)?

You’ll be dreaming of being in line at the airport and the person behind the counter will be exchanging your passport for ice cream, and on your left a gorilla will be taking away your baggage, and all of this will seem totally unremarkable. How come we never stop and go “wait, I wouldn’t expect there to be a zoo inside an airport”?

I have a similar question for when we’re awake: why are we never surprised by the absurd non-sequiturs in our trains of thought? You’ll be eating chicken nuggets, thinking about which TV show to watch, and suddenly remember that you forgot to respond to your friend’s message three days ago, and it will not appear at all strange to you that this thought entered your mind out of nowhere.

A friend who’s a neuroscientist pointed out that the answer here might have to do with working memory. When a thought to respond to your friend’s text enters your mind unprompted, it pretty much displaces whatever else was there, so you momentarily forget that you were trying to figure out what show to watch. (And then you only remember again once you notice the TV in front of you.) Note that working memory is extremely limited.

When dreaming, we have even less working memory, which leaves us with less room to make the observation that there is no legible connection between what happened one second ago and what’s happening now. Another general property of dreams is that your higher-level cognitive faculties (system 2?) are less functional, so you can’t think critically about what’s in front of you.

Q: Why is it that as we learn vastly more facts over our lifetime, the “lookup time” for any specific fact or memory remains the same?

Although we forget many things, the facts we do remember are all still ~instantly accessible, even when there is ten times as much of them as there was before.

I was surprised to discover that we have not found any upper limit on the capacity of our long-term memory. Which makes it even stranger that lookup is still so fast. Like if you imagine a database on a computer, the more entries that database has, the harder it will be to search for the specific item you want from the database. Of course, this all depends on the implementation of the database—there are ways to organize it ahead of time to make lookup really fast. Which then just begs the question of how the brain organizes information to make retrieval efficient.

Q: How does the brain “compute”?

An individual neuron can fire signals every millisecond at most, and it takes at least 10ms for one neuron to transmit a signal to an adjacent neuron, and often longer.2 How does the brain do things like register a unified experience of the world within 500ms, which is something computers still can’t do with processors that are a million times faster?3 The brain seems to be making up for this with enormous parallelism: while your laptop might have about a dozen CPU cores, the brain has a hundred billion neurons.

Of course, this question makes the assumption that the only kind of “computation” the brain does is at the level of connections between neurons, which I’m not sure is true. For example, there is good evidence that neurons are not the only kind of cell in the brain that communicates with electrical signals—astrocytes do too. Also, once you read more about the inner workings of neurons you realize that an individual neuron is doing many things that could conceivably be described as “computation.”

Q: How come the brain, composed of a hundred billion neurons all distributed at different points in space, and thus needing many milliseconds to propagate signals to each other, is able to generate the experience of a single, unified, momentary “now”?

This is a version of the binding problem. Despite reading several books and papers on consciousness I don’t have a good idea of where to start with this one, but Andrés Gómez-Emilsson is converging on an interesting answer in this video and paper.

Thanks to Amanuel for notes on an earlier draft.