Hi Mike,

Having been through several conscious loops of over-riding and reconfiguring “valence” at many levels (in training for deep diving, in training for survival in very cold conditions, in surviving a terminal cancer diagnosis, and others) it seems clear to me that while valence may start with relatively simple biochemical roots, it can actually become a very complex function in and of itself. It seems to me, from about 50 years of active investigation on myself (over 40 of those within the context of having a very high probability that indefinite life extension was a real possibility that might be available in my lifetime, and from my interests in biochemistry, AI, logic, computing, complexity, evolution and life more generally), that valence can itself become a recursively complex system with many modifier functions.

I am now very confident that the “physics” of consciousness is essentially a software on software thing – a software entity existing in a subconsciously created software model of reality, and of course all software is influenced on the “hardware” it is running on, and we have many levels of highly evolved hardware systems within us. So it seems clear to me that it is a very messy, very complex, highly recursive set of systems. And having been writing computer systems for over 40 years, I know how easy it is to create systems whose behaviour is not predictable by any method faster than simply letting them do what they do – maximal computational complexity has that messy property. I like both Jo Ito’s and David Snowden’s approaches to such things.

Thanks Ted. I agree that the complexity of the brain is off the chart- and that reward/value in the context of neural computation is a messy, multivariate, multi-level property.

The question, however, is whether both the brain *and* the fundamental mechanics of conscious systems are complex, or whether the brain is complex but consciousness (like other properties in physics) is starkly elegant, and perhaps not so intrinsically complicated.

My intuition is for the latter- that the brain is horrifically complex, but the physics of consciousness (and by implication valence, a.k.a. what makes a given moment of conscious experience pleasant or unpleasant) could be both simple and tractable.

I’m not *sure* of this, of course, but it seems like if we assume the problem of consciousness / valence is too complex for humans to solve, then we *definitely* aren’t going to solve it.

It seems that reward or value for humans derives from multiple overlapping and intersecting and deeply nested sets of survival heuristics encoded at both genetic and mimetic levels (both cultural and individual).

There are not, nor can there be, any absolute guarantees in life. And we can certainly move a lot of probabilities a long way from where they are right now.

While we hold on to markets, and exchange values, we are in grave existential risk territory (with or without AGI) – see http://tedhowardnz.wordpress.com/2016/05/27/money-cartoon/ for my latest thoughts on the issue – there are many others on that site.

The basic purpose of a utility function is to specify what a system’s goals are, and how to achieve them. The two ways that I can think of to build a utility function are:
– explicitly, via a high-level state variable. E.g., we can define a “utility” variable, increment it for every paperclip the AI makes, and tell the AI to try to maximize this utility variable.
– implicitly, via a dynamic principle. E.g., a thermostat might be set to 70 degrees. If it’s below 70 degrees, the thermostat increases the heat; if it’s 70 or below, the thermostat decreases the heat. We can abstract this behavior and talk about the thermostat as if it has a utility function (even though, technically, it doesn’t really).

Humans definitely do not have utility functions- we’re much too multipolar, irrational, satisficing, etc. to model this way. The closest we come to one is how we sorta-kinda try to increase our happiness/meaning, but you and I both agree that humans are not happiness maximizers in any real sense!