My knowledge sits in a box. Attached to my neck and protected by bone. Mine to open and free to take. Any thought I acquire, ready to be shelved. Except as I walk down the depths of my mind some things are too dark to see.

Sometimes what gets dark, I forget to pick back up. The experiences of infancy, the many breakfasts I ate, the many words I’ve spoken, never to be found again. It’s almost as if my knowledge is not a box at all but a tub with a leak, albeit with a faucet on.

However, my knowledge is not all water. Sometimes what drips in feels different. It remains on the surface never mixing. In fact, my knowledge takes many forms from water, to oil, to gas, to air, to rocks, to people. My knowledge is not a chalice to be filled but a vibrant mix of many conflicting contradictions ill-posed to be interchangeable. Each thought fighting for space trying to survive before their eventual decay. A competition where only the fittest survive. My mind is a market of ideas.

In their ruthless expansion sometimes my ideas cooperate, finding ways to survive together where one cannot. Sometimes my ideas resonate and give birth to something brand new. My knowledge is not the sum of the ideas but the collective whole of interactions of it’s parts. My knowledge is an ecosystem, competing, cooperating, coexisting, and contradicting.

My knowledge is vast because it is an ocean, a forest, a sky, a tree, a house, a friend. My knowledge is my niche in space and time. Knowledge doesn’t fit in a box, a tub, a market, a hard drive, or a brain. Knowledge is the world in which I live. Living is to know.

Dualistic approaches rely on a fundamental distinction between mind and matter. In turn, this class of definitions creates a mind-body problem. How could it possible that an immaterial mind could ever have an effect on a material substance like a body? Although, chocolate vanilla swirl is popular with more traditional thinkers and most religions, alternatives have been proposed. There tend to be three common choices:

• Idealism where everything is mental and matter is a hallucination.
• Neutral Monism which means neither mental nor material are the whole picture and there is something in between.
• Materialism where its matter all the way down.

For the working cognitive scientist materialism or a watered down neutral monism are sort of a job requirement. To be able to experimentally understand mind it needs to be material or very close to it. So how does a materialist (or watered down neutral monist) answer the question “Where is my mind?”

To Eliminate or Not

First though, what the hell is a mind? Usually mind is left undefined and can range from memory to consciousness. The most general definition we can give (in this context) is that it is a collection of mental faculties. There are many mental faculties, we can’t list them all but we can give some examples: emotions, thoughts, reasoning, memory, perception, planning, … Hopefully that gives a good idea of the kinds of stuff that make up minds. One problem though is these faculties were identified well before science had anything to say about minds. It turns out that these individual components are not as helpful as we would like. They are hard to identify and seem to have non-separable physical correlates. In simpler terms, they are too fuzzy to do science with. Although some scientists still hold on to [the modularity hypothesis] (https://plato.stanford.edu/entries/modularity-mind/) proposed by Fodor which claims that some of them are firm enough to build a science upon. However, this leaves us with a choice. Do we keep these terms and hope they are clear enough or do we start from scratch and reconsider what a mind is entirely? The second option I call the [elimination hypothesis] (https://plato.stanford.edu/entries/materialism-eliminative/). Eliminativists argue that we need to divide mind up in new ways and thus eliminate the existing faculties from the scientific vocabulary. This would have a whole bunch of implications, the central one being we as scientists now have the choice to pick what physical systems we consider to be the mind.

Neurocentrism (Mind-Brain Identity)

The simplest object we could pick is the brain. It’s an organ in our head that is always working and seems to be important for a lot of things. Its connected to every major system in our body and seems to correlate with intelligence across species. What if we call the brain our mind? The brain is as good a candidate as any. We found the digestive organ (stomach), the breathing organ (lungs), why not let the brain be the thinking organ. My mind is right here, in my head. Neurocentrism is one of the most dominant ways of thinking not only in academia but in most atheistic approaches to mind. Neurocentrism is also the work that underpins most of neuroscience as a whole. One limiting factor though is that pesky elimination hypothesis. If my mind is my brain then where are my thoughts? Neurons don’t look like thoughts, they also don’t act like them. So I can have this cool definition of mind that is material but I have to give up on explaining questions I might traditionally ask, like “what is love?”, “why am I happy?”, “how do I learn?”.

Embodiment

To motivate this approach lets consider the question “what is love?” A common story of love in the brain is that it is the chemical oxytocin. It is true that oxytocin correlates with what we would call love in many situations but it also correlates with a bunch of other stuff that definitely is not love. Since we’ve already accepted the elimination hypothesis though this is okay, we simply must accept that love isn’t a physical thing and oxytocin is. Love won’t be a scientific question. But what if I really really really want to ask about love, scientifically. Maybe just maybe love is oxytocin plus something else. We could consider how oxytocin interacts with blood pressure for example since love is also correlated with higher heart rates. Now we can start to see when oxytocin is and isn’t acting like love, it depends on whats going on in the rest of the body. This is what embodiment researchers advocate for. They argue that love is just one example but that mind itself can be better understood if we not only consider the brain but the whole body. In this case what makes up my mind is my entire body. There is no thinking organ thinking is what happens in all my organs as they interact together and keep me alive.

Situatedness

We now have two competing options for those willing to accept eliminativism. We can expand our definition of mind to encompass the body which allows us to more accurately capture some of the things we would traditionally associate with minds. Some take this even further. The argument goes that much of what we consider mind is not in one spatial location like a brain or body. Lets consider two of the fundamental functions of mind: action and perception. We could argue that action and perception are impossible without a world. The world has to be present for either of these two functions to be realized. Therefore the world is necessary for mind. Even if we put the brain in a vat we would still need to electrically stimulate it for it to do anything. This means that to have a description of mind we need it to be situated inside the world. Thus we introduce the situatedness of mind. This is the perspective that the world we live in with its physics and chemistry and culture and society is in a sense part of our mind. There both more moderate and more extreme versions of this position although it is probably one of the harder ones to commit to. However, it has lead to one of the most successful [research programmes] (http://psychsciencenotes.blogspot.com/2017/04/the-ecological-approach-explained-to-8.html) in all of psychology.

Concluding Remarks

Here I have presented some arguments for eliminativism as well as embodiment and situatedness. I don’t believe any one of these perspectives is exactly right, although personally, I think situated and embodied cognition is the most interesting. One conclusion worth drawing is that science is about creating new ideas just as much as it is about discovery something that exists. There is no pure discovery. Every scientific idea is built on a world of assumptions. Making those explicit allows us to try new ones and maybe find something better. The universe is messy and doesn’t fit well into our little boxes. That doesn’t mean we give up on boxes but it does mean we need to be okay with changing them and moving stuff around once in a while.

For this post I want to consider the examples we use in the study of intelligence and how different examples may emphasize different parts of thinking. A popular definition of intelligence either implicit or explicit is that it is the ability to solve many problems well. This definition is far from complete and needs work, but from it we can see that a study of intelligence would require us to decide on a set of problems and define what “well” means. One context in which this is easy is games. Games are places with many problems, they also have clearly defined rules and thus being good or doing well is clear. Games also serve as context in which we can compare our intelligence to the intelligence of machines relatively easily. Much research in the fields of AI and cognition has been done on games. So let us consider two games: Chess and Dead Cells.

Chess and Computational Mind

Chess is played on a board with 64 tiles. In the world of chess player 1 makes a move, player 2 makes a move, and so on. There is no parallelism no timing, just looking at the board and making a decision.

I want to make an AI that plays chess. How could I do it? This is where symbolic manipulations or computationalism comes in. Anyone who has played chess has probably “thought a few moves ahead.” In the world of algorithms we call this process enumeration. I consider the consequence of one move, if I like it I make it, if not I look at the consequence of the next move and so on. This is a relatively straightforward way to solve a problem. I look at my options one by one and pick the best one. For chess and many other daily tasks this is often how I find myself thinking. What’s nice about this way of thinking is how easy it is to implement on a computer. Anyone whose written a for loop before can understand the basic idea.

One limitation is that this is not always possible. Even in chess I can’t look at every possible sequence of moves; that would take too long. However, I can look at the most likely moves. If I have played chess a lot, I know which moves are more common than others. I can use probabilities to make rational choices. This is more realistic because sometimes I make mistakes. A mistake simply means I had the probabilities wrong. By using chess as my example I emphasize certain aspects of play and this results in certain ways of thinking. Namely, I think of the symbolic manipulations of abstract pieces rather than how a hand moves a knight. The specific way in which I move my body, the exact amount of time I spend thinking, the way in which visual information plays a role are all in a sense non-fundamental. In fact, I can completely determine how to do well by only looking at the current board state rather than considering historical dependence. The reason I ignore these things is not necessarily because they are unimportant. They are simply not the first things I concern myself with when I want to play chess well.

One thing to notice with this way of thinking is that humans are somehow capable of doing what a computer does. We can make a list of moves, we can calculate probabilities, we can make “rational choices.” This is a result of picking a game in which rationality is well defined. The rational move is the one that brings you closest to your goal and those moves are simple elements from a discrete set.

Limited Perspective

The above example illustrates a family of philosophy of mind called computationalism. In computationalism, thinking is what a computer does. Humans also think but our hardware is the brain instead of a computer. If your goal is to understand how humans play chess then maybe this is a good way of framing about the problem. However, to think in these terms we make some simplifying assumptions: the fact that I have to move a piece with my arms and hands is ignored. The fact that I have to move my eyes around the board to see all the moves is ignored. The fact that my thoughts have to be very quick and efficient is generally ignored. Things being ignored isn’t that big of a problem: physicists ignore friction, chemists ignore quantum mechanics, classical economists ignore data. Ignoring things is a big part of science. What you choose to ignore however is still an important question and often depends on your model system. For an alternative let’s consider what we ignore and don’t ignore when we take as our example to be Dead Cells instead of Chess.

A Situated Embodied Dynamical Game

For those of you who haven’t played: Dead Dells it is a 2D platformer/hack n slash game. You can kill things with your weapons and avoid getting hit by enemies. Unlike Chess your interactions with the game are in real time. You need to dodge, attack, jump, and plan your way through to beat a level. This makes it kind of hard to ignore certain facts about the game. You have to consider how quick the system is to act in the environment, how the environment is being navigated, how a previous decision impacts the next one and so on. What you choose to ignore has to be different.

Taking this to be your model system you will notice three things about thinking that may not be as easily apparent when you think about chess. First the position of the character in space matters a lot. This is what we call embodiment. It matters what actions a character can take, what weapon they are using, how high they can jump, and how fast they can travel. All of these features will fundamentally affect how a given player plays the game. Thus the nature of the body is part of the cognitive (thought) process.

The second thing you’ll notice is that it’s not just the body which is used to make decisions. If you watch someone playing Dead Cells they will often dodge into a door. Dodging into a door has the effect of breaking the door. By breaking the door I create a marker which tells me I have already been in this area. I don’t need to remember every place I went to on the map. I changed the environment so that I can use my senses to make that decision rather than remembering it. Kind of like taking notes in a journal. This second property is what we call situatedness. Situadedness means that thinking is not just something we do with our brains and bodies but we also think using our environment. This is also why usually if you are trying to be more productive you can move to a different room. A change in your environment is also a change in your mind.

Usually, when we abstract away concepts, the first to go is time. In chess this is relatively easy the turn order is fixed so time isn’t fundamental. We can always solve the problem and then make it faster. If you play Dead Cells though you should know this isn’t possible. Timing is everything. Whether you make it on that platform or dodge that attack you need to time it right. If you pause the game in the middle of an intense battle and take a break chances are you won’t survive when you start back up. You will have lost momentum. This is I believe to be one of the most important parts of thought. When we think we are not independent of time. Our thoughts happen in time. This is the dynamical aspect of cognition.

Dynamics means things that change over time. The dynamics only become apparent when we look at certain games to be our model system. As most gamers know however, modern games require a intuitive and skilled use of timing. If you ignore time it can be easy to think the cognitive work can only be distributed over space. Which in some sense it is. But it is crucial to understand that cognitive work not only extends over space but also time. The moments between moments can themselves have computational value which means that the system can embed calculations across time and thus the relevant dynamics can emerge. This is a fundamental shift from the computationalist view of mind. What makes calculation and computation a useful metaphor for thought is that they give us objects to look for when studying minds. However, in some cases the computations are so distributed and abstracted that they may not even be identifiable in the first place. Depending on our model system computation may no longer be a good metaphor for thought at all.

Conclusion

So now we know that thinking can happen across bodies, the environment, and time. From here we should start to a see a picture emerging. Intelligence is not something that we can say happens in one spot in one moment. There is no thought I can point to in my brain. Instead, thought is continuously happening all over the place. This shift should make us question how we design our experiments and how we build our models. Most importantly however, it should make us realize that our examples are only examples and it is after we have looked across many examples that we can start to see what truth actually looks like.

The problem comes from many interacting norms and conventions (scientific and political). But it can be stated simply. We think there is a mind between perception and action. The world passes through perception to the mind where a decision is made and a motor program is executed. This is bad for serveral reasons. The passage of world through perception loses accuracy and thus must be predicted. The decision making process must be fast and able to handle vast ammounts of data. The motor program must be perfectly suited to deal with the fact that muscle activation is context dependent. The whole idea is precarious and expensive.

What makes people different? People live. Living is one process. That process involves a brain, a body, and an environment. There will be attributes of the environment that are relevant to the organism, the organism has evolved such that its behaviors reflect those patterns. I walk on grass and can climb a tree. I know I like sweets and mom’s voice makes me feel safe. This is product of a being alive in the world. We don’t recreate the world to make a virtual version so that we can predict reality. We are not solipsistic we don’t hallucinate the world. We are of the world. We experience it and in that experience there are properties of the world that affect us. That is perception. It is to be affected by the world.

This was Gibson’s notion of direct perception. There are robots inspired by these principles. It’s what gave Boston Dynamics the edge to build killer robots for an evil government. So that’s cool but also it gives us a new way to think and when the time comes build better robots.

A Study of Mind

Like most neuroscientists, cognitive scientists, and AI researchers, I am interested in minds. The problem is no one knows what a mind is. Philosophers have been working on this question for millennia and we still don’t know when something has or doesn’t have a mind. Scientists being a little too busy to care about a correct definition, have come up with their own definitions based on what they each find most interesting. Letting a thousand flowers bloom tends to work well, so I support it. However, for any scientists out there, it doesn’t hurt to read some philosophy once in a while.

My definition of a mind is mainly inspired by the Biology of Cognition or Santiago School. The basic idea is that minds are a property of life: that it can “make its own decisions.” This feature of life is hard to pin down exactly but becomes most obvious when we look at how organisms survive. Animals don’t move randomly, they navigate to food, avoid predators, find safety. They don’t behave based on fixed rules but instead adjust their behavior to improve their chance of survival. This is also sometimes called adaptive behavior. For me this is the foundation of what it means to have a mind. In adaptive behavior we see goals, intentionality, and values. Seems like a good place to try to study minds.

A Science of Behavior

The study of animal behavior has existed for a long time. Charles Darwin himself spent much of his time travelling and examining how animals act. Ethology is the term used for the biological side of animal behavior research. Ethologists promote studying animals in natural habitats preforming autonomous behavior. This differentiates it from most other fields of animal research, where animals are kept in highly controlled environments solving very restricted problems. The ethological argument goes that animals evolved in their natural habitat to do what they do and to understand them we must observe them in that context.

Taking Behavior Seriously

One limitation of traditional ethology was that it remained descriptive. Much of the discipline was focused on building a library of behaviors rather than tackling what unified all of them. How behavior is coordinated was almost completely ignored. In response the field grew a new appendage. One which added the toolkit of neuroscience. Neuroethology is a discipline concerned with the relationship between brain and behavior. Neuroethologists not only study how an animal acts and moves but also what is going in the animal’s brain when that movement happens. In this way neuroethologists try to provide a mechanistic description of how action and perception together create behavior.

What about the Computers

Neuroethology is a small niche but one advantage of the label is its lack of preconceived meaning. Computational on the other hand means a lot of things. The computational component of computational neuroethology might best be understood in relation to computational physics. When quantifying phenomena in the world sometimes things get complicated. When a physicist is curious about how a fluid flows, they do not do the math by hand. They use a computer to simulate the many complex equations as they interact. This is the computation in computational neuroethology. Except rather than simulating particles I simulate brains, bodies and environments. I build video game worlds and populate it with virtual animal bodies connected to virtual brains. Luckily AI can serve as the virtual brain and robotics provides virtual bodies. I then compare how my virtual robot behaves to how the animal behaves. By doing this repeatedly and building different worlds and different brains and different bodies we might understand what unifies all adaptive behavior: A grand theory of thought.

Final Words

In short, my day to day work is playing make believe with video game animals. I am optimistic that it will lead to more than that though. My goal is to help contribute to a formal theory of minds. Such a theory would not look like the human-centered, rationalist, or psychological approaches that have dominated our historical understanding. Instead, I believe it will show us how genuinely unexceptional humans really are and how special life itself is. A theory of thought that will teach us that there is a continuity between us and every one of our relatives on this planet. A continuity not only in our historical lineage but a true recognition that all life has wants and desires and rich inner lives. Most optimistically I hope to help build a theory that reminds us that we have so much to learn and unless we listen we will lose it all.