Priorities, Hibernation and Twitter

To the few readers of this blog,

I have an important task ahead of me that requires most of my attention, time and energy. That is, completing my PhD will need to be my priority for the next 6 months or more.

Few people are likely to miss my already infrequent posts, but they are likely to become more infrequent. Once my PhD is complete, I hope to embark on some larger projects, posting on the philosophy of mind, the meaning of life, and artificial general intelligence. I will still try to read the posts of other bloggers and comment when possible.

In the meantime, I take public transport, so I may use this time to take up tweeting with more frequency. Sometimes I will tweet links to interesting articles, but I also hope to start tweeting some of my ideas and thoughts.

Questions and comments are still welcome on my blog, but responses may be slow.

[smiley face]

Learning Algorithms for People: Reinforcement Learning

In a previous post I described some simple ways in which learning could be performed algorithmically, replicating the basic process of supervised learning in machines. Of course it’s no secret that repetition is important for learning many things, but the exact way in which we repeat a training set or trial depends on what it is we are trying to learn or teach. I made a series of post on values and reinforcement learning machines and animals, so here I will describe a process for applying reinforcement learning to developing learning strategies. Perhaps more importantly though, I will discuss a significant notion in machine learning and its relationship to psychological results of conditioning — introducing the value function.

Let’s start with some pseudo-code for a human reinforcement learning algorithm that might be representative of certain styles of learning:

given learning topic, S, a set of assessment, T, and study plan, Q
    for each assessment, t in T
        study learning topic, S, using study plan, Q
        answer test questions in t
        record grade feedback, r
        from feedback, r, update study plan, Q 

This algorithm is vague on the details, but this approach of updating a study plan fits into the common system of education; one where people are given material to learn and given grades as feedback on their responses to assignments and tests.

Let’s come back to the basic definitions of computer-based reinforcement learning. The typical components in reinforcement learning are the state-space, S, which describes the environment, an action-space, A, are the options for attempting to transition to different states, and the value function, Q, that is used to pick an action in any given state. The reinforcement feedback, reward and punishment, can be treated as coming from the environment, and is used to update the value function.

The algorithm above doesn’t easily fit into this structure. Nevertheless, we could consider the combination of the learning topic, S, and the grade-system as the environment. Each assessment, t, is a trial of the study plan, Q, with grades, r, providing an evaluation of the effectiveness of study. The study plan is closely related to the value function — it directs the choices of how to traverse the state-space (learning topic).

This isn’t a perfect analogy, but it leads us to the point of reinforcement feedback: to adjust what is perceived as valuable. We could try to use a reinforcement learning algorithm whenever we are trying to search for the best solution for a routine or a skill, and all we receive as feedback is a success-failure measurement.

Though, coming back to the example algorithm provided, considering grades as the only reinforcement feedback in education is a terrible over-simplification. For example, consider the case of a school in a low socio-economic area where getting a good grade will actually get you punished by your peers. Or consider the case of a child that is given great praise for being “smart”. In a related situation, consider the case of praising a young girl for “looking pretty”. How is the perception of value, particularly self-worth, effected by this praise?

Children, and people in general, feel that the acceptance and approval of their peers is a reward. Praise is a form of approval, and criticism is a form of punishment, and each is informative of what should be valued. If children are punished for looking smart, they will probably value the acceptance of their peers over learning. If children are praised for being smart, they may end up simply avoiding anything that makes them look unintelligent. If children are praised for looking pretty, they may end up valuing looking pretty over being interesting and “good” people.

A solution could be to try to be more discerning about what we praise and criticise. The article linked above makes a good point about praising children for “working hard” rather than “being smart”. Children who feel that their effort is valued are more likely to try hard, even in the face of failure. Children who feel that praise will only come when they are successful, will try to avoid failure. Trying to give children self-esteem by praising them for being smart or pretty is, in fact, making their self-esteem contingent on that very thing they are being praised for.

It may seem manipulative, but praise and criticism are ways we can reward and punish people. Most people will adjust their “value-function”, their perception of what is valuable, and, as a result, they will adjust their actions to try to attain further praise, or avoid further punishment. What we praise and criticise ourselves for, is a reflection of what we value in ourselves. And our self-praise and self-criticism can also be used to influence our values and self-esteem, and hence our actions.

A View on the Nature of Consciousness

In the process of communicating with other bloggers that are interested in the future of humanity and philosophy of mind, some upcoming discussions have been planned on a number of related topics. The first topic is: the nature of consciousness and its relationship to the prospect of artificial intelligence. In preparation for the discussion, I’ve summarised my position on this topic here. I’ve spent some time reading and thinking about the nature of consciousness, so I believe that my position has some firm evidence and logical reasoning supporting it. If requested, I’ll follow up with post(s) and comment(s) providing more detailed descriptions of the steps of reasoning and evidence. As always, I’m open to considering compelling evidence and arguments that refute the points below.

The Nature of Consciousness

1. Consciousness can be defined as its ‘nature’. We seem to define consciousness by trying to describe our experience of it and how others show signs of consciousness or lack of consciousness. If we can successfully explain how consciousness occurs — its nature — we could then use that explanation as a definition. Nevertheless, for now we might use a broad dictionary definition of consciousness, such as “the awareness of internal (mental) and external (sensory) states”.

2. Consciousness is a spectrum. Something may be considered minimally consciousness if its only “aware” of external-sensory states. Higher consciousness includes awareness of internal-mental states, such as conscious thoughts and access to memories. Few animals, even insects, appear to be without “awareness” of memory (particularly spatial memory). As we examine animals of increasing intelligence we typically see a growing sets of perceptual and cognitive abilities — growing complexity in the range of awareness — though varying proficiencies at these abilities.

3. Biological consciousness is the result of physical processes in the brain. Perception and cognition are the result of the activity of localised, though not independent, functional groups of neurons. We can observe a gross relationship between brain structure and cognitive and perceptual abilities by studying structural brain differences animal species of various perceptual and cognitive abilities. With modern technology, and lesion studies, we can observe precise correlations between brain structures, and those cognitive and perceptual processes.

4. The brain is composed of causal structures. The collection of functional groups of neurons in the entire body (peripheral and central nervous system) are interdependent causal systems — at any moment neurons operate according to definable rules, effected by only the past and present states of themselves, their neighbours and surroundings.

5. Causal operation produces representation and meaning. Activity in groups of neurons have the power to abstractly represent information. Neural activity has “meaning” due to being the result of the chain of interactions that typically stretch back to some sensory interaction or memory. The meaning is most clear when neural activity represents external interactions with sensory neurons, e.g., a neuron in the primary visual cortex might encode for an edge of a certain orientation in a particular part of the visual field. There is also evidence for the existence of “grandmother cells”: neurons, typically in the temporal lobe of the neocortex, that activates almost exclusively in response to a very specific concept, such as “Angelina Jolie” (both a picture of the actress and her name).

6. Consciousness is an emergent phenomenon.  Consciousness is (emerges from) the interaction and manipulation of representations, which in biological organisms is performed by the structure of the complete nervous system and developed neural activity. Qualia are representations of primitive sensory interactions and responses. For example, the interaction of light hitting the photosensitive cells in the retina ends up represented as the activation of neurons in the visual cortex. It is potentially possible to have damage to the visual cortex and lose conscious awareness of light (though sometimes still be capable of blindsight). Physiological responses can result from chemicals and neural activity and represent emotions.

7. Consciousness would emerge from any functionally equivalent physical system. Any system that produces the interaction and manipulation of representations will, as a result, produce some form of consciousness. From a functional perspective, a perfect model of neurons, synapses and ambient conditions is not likely to be required to produce representations and interactions. Nevertheless, even if a perfect model of the brain was necessary (down to the atom), the brain and its processes, however complex, function within the physical laws (most likely even classical physics). The principle of universal computation would allow its simulation (given a powerful enough computer) and this simulation would fulfil the criteria above for being conscious.

8. Strong artificial intelligence is possible and would be conscious. Human-like artificial intelligence requires the development of human-equivalent interdependent modules for sensory interaction and perceptual and cognitive processing that manipulate representations. This is theoretically possible in software. The internal representations this artificial intelligence would possess, with processes for interaction and manipulation, would generate qualia and human-like consciousness.

Philosophical Labels

I’ve spent some time reading into various positions within the philosophy of mind, but I’m still not entirely sure where these views fit. I think there are close connections to:

a) Physicalism: I don’t believe there is anything other than that which is describable by physics. That doesn’t mean, however, that there aren’t things that have yet to be adequately described by physics. For example, I’m not aware of an adequate scientific description of the relationship between causation, representation and interpretation — which I think are possibly the most important elements in consciousness. Nevertheless, scientific progress should continue to expand our understanding of the universe.

b) Reductionism and Emergentism: I think things are the sum of their parts (and interactions), but that reducing them to the simplest components is rarely the best way to understand a system. It is, at times, possible to make very accurate, and relatively simple, mathematical models to describe the properties and functionality of complex systems. Finding the right level of description is important in trying to understand the nature of consciousness — finding adequate models of neuronal representations and interactions.

c) Functionalism: These views seem to be consistent with functionalism — consciousness is dependent on the function of the underlying structure of the nervous system. Anything that reproduces the function of a nervous system would also reproduce the emergent property of consciousness. For example, I think the ‘China brain’ would be conscious and experience qualia — it is no more absurd than the neurons in our brain being physically isolated cells that communicate to give rise to the experience of qualia.

Changing Views

I’m open to changing these views in light of sufficiently compelling arguments and evidence. I have incomplete knowledge, and probably some erroneous beliefs; however, I have spent long enough studying artificial intelligence, neuroscience and philosophy to have some confidence in this answer to “What is the nature of consciousness and its relationship to the prospect of artificial intelligence?”.

Please feel free to raise questions or arguments against anything in this post. I’m here to learn, and I will respond to any reasonable comments.