Determination 4 (Simplicity)

Before defining a "decision" in the context of a deterministic system, I want to talk about simplicity.

There are literally an infinite number of explanations for everything we observe.

You might be watching a stage play, and see an actor fly onto the stage by what appears to be some mechanical contrivance. (You don't think you're watching Superman's cousin as a thespian, after all. Although that legitimately is one of the infinite number of explanations that might explain what you're seeing...) If you know a little bit about the sorts of gears and pulleys that might make this work, what are you going to imagine they look like? What's your favored explanation of how they got this character flying? You will probably imagine some relatively simple setup. You will probably not imagine that the gear-work that is being used to fly the human body around does not have the primary purpose of predicting future stock prices and economic volatility.

The choice will be the relatively simpler explanation over the relatively more complex explanation. What you won't expect is to get the answer entirely right.

But we need some way of sifting through that infinity of choices, and the main principle that we always seem to land on -- as conscious but ignorant agents inside a big mysterious world -- is some notion of simplicity. As we start to enumerate the list of possible explanations for any given observation, what we find is that the more explanations we list, the more convoluted the explanations become. It's not just that the infinity of explanations is (by definition) quite numerous, it's that the infinity of explanations become more and more complex as we make the list longer. And what that necessarily means, if we go in the other direction, is that as we enforce some notion of simplicity on our list of choices, we end up cutting away that hazardous infinity. We bring order to the list by focusing on the simplest choices. As we get simpler and simpler explanations that are -- very important -- still sufficient to explain the given observation, we might even find that there is one explanation that reigns supreme over all as the simplest. It doesn't work that way in the opposite direction. There is no "most complex" explanation because more complexity can always be added. But there is (potentially) a most simple explanation, one that stands in isolation apart from all others based on its parsimony.

This is the idea behind the razor.

This is a fucking powerful idea. It's a method of cutting away the infinity of explanations to leave just a few guesses which, although perhaps not fully correct, seem "more likely" to be correct because they are the cleanest and most elegant explanations that we have. But okay, clean and elegant, that's all very fluffy. It's not rigorous. What does that mean? Well, the task at hand is to define "simplicity" in a way that is as fully plain and rigorous and unambiguous as we have previously defined determinism.

What we DON'T want is to rely on human intuition for this rigorous definition of simplicity. The human brain is an evolved organ. It's very, very good at facial recognition. It's very, very bad at intro probability questions. This is not because intro probability is "complex" in some objective sense. It isn't. Looking at a face in order to understand whether the person is angry at you is an enormously complex computational problem -- but it is also utterly necessary for survival if you don't want to get a spear in the guts. Anger, joy, disgust. These are simple emotions. We "understand" all of them very easily. But biologically/neurologically/computationally, we don't understand how any of this stuff works. We need a more objective notion of "simplicity" if we are going to rely on that notion to decide -- if even only provisionally -- between the different explanations we have created for the observations in front of us.

Math comes to the rescue.

We actually do have a fully rigorous notion of simplicity that comes direct from the world of information theory. The basic notion here is that you attempt to write a computer program to fully describe the observations that you're seeing.

Think about a screensaver with a bunch of colorful balloons bouncing around the edges of the screen. What would be the simplest possible program that could run that screensaver? Sure, the screensaver might be calculating the digits of π in the background while you can't see. Or it might be forecasting future weather patterns and just not showing those calculations on the screen. Or it might be subject to random discontinuous jumps that happen every 10,000 years in run time. Or it might be running an eternal simulation of a game of Civilization 2. Or it might be simulating another set of colorful balloons that orbit the center of your screen, but at a distance of around 10 feet, so that the program is always keeping track of those extra balloons, but you never see them because the orbit never brings this extra set of balloons within the monitor's field of vision.

There are an infinite number of these explanations, but if you're going to try to decide on your best guess of which one is correct, what guess on you going to land on? Do you really think the screensaver is running an economic model in the background when you're not looking? Really? If you had to choose your one best guess of what the program is actually doing, what it really looks like, would you err on the side of more and more and infinitely more complexity, continually adding more things the program might be doing in the background while you're not looking?

Or is your best guess that the program is relatively simple?

You're not going to be 100% sure that your best guess is correct. Of course not. But if you're looking at more complex guesses, vs simpler guesses, you will -- without any doubt whatsover -- be limiting your focus the finite subset of the simpler possible guesses, rather than continually considering more and more and more complex guesses out into infinity. (And we're still talking about a literal infinity here. You might say that the computer doesn't have the resources to run an infinitely complex program... but how do you actually know that? Maybe it does, and you just don't realize it.)

This entry was posted in Uncategorized. Bookmark the permalink.