# Noise in the Machine: The Homogeneous Chaos Blues - 2

### By Jim Chaffee

If you’re still along, grab a padded, well-secured seat and clutch something substantial. We face a wild-ass ride coming right up, so irregular it will shake loose anything not bolted down. Can’t be helped, but you might like it. An honest-to-God Coney Island of the mind, so to speak.

In 1827, English botanist Robert Brown noted peculiar behavior of small particles suspended in fluid. They were, it seems, floppy, twitchy and unpredictable. This jitterbug waltz gained the title Brownian motion. And though this erraticism came to be considered a manifestation of molecular motion, it wasn’t until 1905 that Albert Einstein invented a suitable theory. Said kinetic theory got itself amended in the physical world later, since Einstein’s own explanation was not completely satisfactory, leading to infractions of one or two laws of nature. The fun part came, however, not from any considerations of natural-law abiding physicists but from lawless mathematicians, beginning with Norbert Wiener.

Norbert ran with this illegal idea and created from it a fantastical creature representing Einstein’s logically wanting physical theory. (This academic behavior is a source of irritation to physicists who cannot understand why it is necessary to create an elaborate ruse to make logically inconsistent ideas meet the rigor demanded by the mathematical artist. The most famous example might be Paul Dirac and Laurent Schwartz. Perhaps Emerson’s phrase consistency is the hobgoblin of little minds ought be the motto of physicists.) What Norbert created lived in an infinite dimensional space of functions, and was indeed a way of measuring chunks of such large places, albeit in a rather roundabout way for the likes of most of us. He gave it names like the homogeneous chaos (which has nothing to do with the chaos alluded to above, and in fact predates it by half a century more or less), and his process came to be known as the Wiener process, though a lot of mathematicians still call it Brownian motion, being dubious of, or at least unconcerned with, physical reality.

And Norbert made the engineer’s white noise a living mathematical fiction via his construction of Brownian motion.

Literary types love terms like white noise, though they seldom have a pig’s-eye view of what the hell it is. To be somewhat carelessly careful, white noise is a random process wherein events are independent of one another, no matter how close together they may occur in time. In other words, if you see what this guy did at some time, you will have no idea what it will do next or any time later. This white noise immediately forgets where it was. This implies for the engineer what is called a spectrum that is constant across all frequencies, which means in practice that such a process requires infinite power to run it. That’s right: if you plugged your white noise machine into the wall socket you’d drain all energy that ever had been or would be created, and it would still not run. Which means you can’t trot down to Best Buy and find one of these babies on the shelf. But it represents in some sense the most random behavior to be found in the universe. And it obeys laws.

The law it follows when its paths are summed (meaning integration, for those who have a smattering of education, be it formal or self) is Wiener’s process, the fictive pure Brownian motion without physical impediments like drag. Of course, as noted previously Einstein broke the law with this creation since it had no bounds on its velocity. In fact, it turned out to be so irregular in its trajectories, if you can call them such, that it covers an arbitrarily large distance in any finite interval of time. And all the while with hands never leaving the body, so to speak; that is to say, always turning sharply while remaining continuous. So of course this perky speed demon refused to obey the speed limit of light. Note the adjective for the trajectories was continuous, not smooth. Because these paths are so jagged they have corners on the corners, you might say. For a mathematician this means the paths have no derivatives anywhere, which is to say that if you look at the process’s path differences over disjoint intervals of time, they will be independent of one another. If you consider smaller and smaller such disjoint time intervals, you begin to see where white noise gets its bad behavior. You can observe this process for any amount of time you wish; the observation is useless because the process will start from the last observed spot as if it were born afresh in that moment.

And so you ask, How the hell can such a thing as this obey laws? Much like politicians and corporate executives (an interchangeable lot, actually) obey laws. Like when I wrote above that the integrated white noise trajectories are Wiener process sample paths; though formally true, that was a lie. There are a couple lies; one simple due to Wiener and one more complex due to Kyioshi Ito that was made intuitive by Steve Rosencrans in some course notes. Think smoothing out second-order wiggles, at least in the Rosencrans experience of Ito. Still pretty jagged, even at that, and giving rise to pesky second-order wiggles engineers ignore at their peril, especially when unaccounted for said second-order wiggles fuck up the extended Kalman filter’s gain, causing it to crash and burn.

And now as we finally peak, take a gander down upon erose terrain and behold an endless landscape of bottomless crevices and twisted precipices. Terrifying. But in the aggregate tamed. Because if we have enough riders on these trails, we can average them and get deterministic. That’s right. This nearly totally random monster (Gaussian, for those who care about such things) of Wiener’s via Einstein provides us a solution to a deterministic problem. Bound some region nicely and give it some functional preconceptions meeting a smattering of conditions and then release a passel of these maniacal wrigglers, capturing them as they try to cross the perimeter and averaging them with respect to those conditioned preconceptions and you solve a classical deterministic partial differential equation.

How the hell does this happen? Well, let’s skip the long story and just say that the friendly little BM (as the Brownian motion in its incarnation as Wiener process is often nicknamed) is related to an operator called the Laplacian that lives in infinite-dimensions. This operator has a long history in mathematics and physics. Moreover this operator can be related to the diffusion of heat, the heat equation which is supposed to model the flow of heat via a partial differential equation. Deterministically.

For example, take an infinite wire with perfect insulation and hit it with an outrageously hot torch at a single spot. Just for an instant. Then the heat distributes along this wire according to this heat equation. But there are some issues. For one, no matter how far away from the torch you hold this infinite wire, at torch touch you immediately feel some heat. That somehow seems wrong, but then no one has ever seen an infinitely long, perfectly insulated wire. What is curious is that the Brownian particle dances in the same manner as the heat distributes, which explains why it is compelled to get so far so fast.

Actually the dance of the BM is described probabilistically by the solution to this heat equation. That is to say, the heat equation lays down the law to this floppy, twitchy, and unpredictable process. So this relationship isn’t so surprising. Moreover, by choosing more general “elliptic” operators than the Laplacian one can get seemingly more exotic dancers. These are called diffusion processes, and what is amusing is the existence of people who apply this to finance. They have been involved in some remarkable disasters with their techniques, notwithstanding the Nobel prize for economics to the inventors.

The unpredictability of the noise process is kind of misleading anyway. One can make predictions but only in terms of the long run. It’s a lot like when a misguided meteorologist claims that because climate scientists cannot predict the short term behavior, they cannot predict the long term either. That is simply false, as work with stable devices such as atomic clocks or gyroscopes demonstrates. Short term forecasts are notoriously bad with noise, but long term trends are better predicted because of averaging. There is a statistical tool developed by engineers for this sort of behavior in such devices, called the Allan variance.

Anyway, it turns out that noise can be used to construct the deterministic, and the deterministic lays down the law to noise. That was what I meant with the idea of weeds and noise: for me, that pesky Brownian motion is not noise so much as just another side of the regular world. Besides solving deterministic problems (albeit inefficiently in a numerical sense), it can also explore both the geometry and the topology of (abstract?) objects resting in arbitrary dimensions.

But for an engineer with a GPS receiver, noise can be a damned weed. And though we have been stuck on white noise, we also encounter what is sometimes called colored or pink noise. This noise has relationships with itself over time. One of the more bizarre variants is the so-called 1/f noise, which is related to fractals it seems. Benoit Mandelbrot is the guy to see about this, and all we can say here is that this noise is very unlike white noise. While white noise has no relationship to its own behavior at any time in the past, 1/f noise has self-relationships that go to the infinite past, whatever that is. It never forgets where it has been, so to speak, whereas white noise immediately forgets. Yet perhaps all these colored noises are the progeny of white noise, but that is way outside the realm of where we ought to go.

Noise permeates everything because so far as I can tell, everything is noise. Unless of course there is the not-noise and not-deterministic. That would be the haphazard, as my old friend and teacher Roger Carlson liked to call it. The opposite of determinism is haphazard, not random. For example, haphazard would be if your neighbors became rhinoceroses; worse, if the E. coli in your gut become rhinoceroses. Small ones, say. Statistical mechanics has no place for such events to occur often, though there is a famous theorem of Poincare regarding events eventually recurring no matter how small their probability (that is, no matter how contrived they may have been originally), so long as positive and you can wait long enough. Sort of a mathematical version of Murphy’s law. But we don’t expect to see it on the macro scale outside Kafka or Ionesco or similar fantasists. Laws of averages hold sway, keeping Wiener’s homogeneous chaos blues away. Hopefully.

[1] Portuguese translation Zimmer's article

[2] Zimmer's article

[3] Bogus-speak is language with the intent of sounding scientific or precise. A terrifying example that has invaded the language is the use of impact to replace the noun effect and verb affect (and perhaps sometimes the verb effect, but this is not clear). Some have argued that this hideous abomination is akin to a fecal impaction of language (mouth turds refusing to budge), much like the use of awesome as an adjective to describe some triviality you might find slightly special. Impacted prose, particularly in the verb case, might be the result of broadcast journalists not being literate enough to grasp the difference between affect and effect, and hence choosing to smear the meaning of a once precise verb as substitute. That this is incorrect can be seen with a bit of reflection: media journalists (and perhaps print journalists, too) are not sufficiently literate to realize there is a difference between affect and effect as verbs (and have been known to use the noun affect when effect was meant).

The real culprit seems to be economists, who in their need for certification as scientists appropriated language and mathematical technique to become a modern cargo cult, emptily parodying physics like ceremonial magic with none of the result. Of course, the excuse is that they cannot control experiments as can physicists, in a lab, though I seem to remember neither Newton nor Einstein were able to bring our planetary system into a lab (or even the Sun and one planet). The actual difference is that when physical theories provide incorrect predictions (and the ability to predict is the hallmark of a theory), physicists replace those theories. Hence the perihelion of Mercury, and relativistic versus classical mechanics. When an economic "theory" makes a prediction, a rare event, and it is wrong, the economist blames reality, not his own ideas. Though these rare events are not easy to come by, consider the beautiful mathematics of Black-Scholes, based on the theory of random processes known as martingales and their integration via Kiyoshi Ito’s formula, and the debacle of Long Term Capital Management. A failure of the "theory" of economic engineering. Not the first, nor the last, given the prevalence of derivatives in our financial web. And the culprit, as it turns out, was reality. Out of step with a mathematical model pawned off as an "economic theory."

At any rate, the testimony of Crocker and Petraeus before Congress provides an instructive example of the use of bogus-speech in its most common form. Words like trajectory are employed to give an aura of determinism, of control, as if things are going as expected along the path to which they have been steered. The terms sound precise, scientific, as if taken from automatic control theory, but here the “trajectory” is the conversion of the new Iraqi government into a satrap of the US, an outpost in which to base troops for the empire. Clearly this is not going quite as determined by the initial conditions, at least as seen from the Pentagon or the Bush White House.

Reality is that bogus-speak is an elaborate form of smoke-blowing. When you hear it in most contexts you can be certain someone is bullshitting you. In its most common form, as before Congress, it pits one or more actors against an appropriately august body divided into two or more sides engaged in a zero sum game for which the speakers are tokens, said speakers comporting with appropriate demeanor presenting appropriate language which no one really understands but designed to make everyone concerned delude themselves and wallow in the obvious lie, or else use the lie as an appropriate ass-cover and perhaps excuse for later mea culpas. It is also used to help other parties not part of the august body, such as citizen-consumers, be at ease with what everyone knows in their heart to be a fucking lie. That is, it is part and parcel of a mass self-delusion.

© Jim Chaffee 2008