Category Archives: learning

On Subconscious Action

from reference 1

The subconscious components of our minds are more powerful than many admit, or feel comfortable admitting. As we go about our lives, our subconsciousness learns about aspects of our existence that might otherwise clutter our thoughts with a distracting chatter of activity: what pressure must I apply to the coffee cup in order for it to remain in my grasp, what route must I take through the throng of commuters in Penn-station to avoid colliding with others, has my blood-sugar dropped below the threshold where I experience hunger, et cetera. In fact, to some extent, the subconscious mind has access to information that the conscious mind does not, as in the muscle tension and blood-sugar examples. Understanding how these abilities are segregated between conscious and unconscious, and to what extent that question even makes sense to ask at both a behavioral and neurophysiological level are of fundamental interest to the understanding of consciousness and human neurological function in general.

A recent study speaks to this topic by probing the extent to which the subconscious can learn about the association between briefly presented visual cues and a monetary reward1. Specifically, Chris Frith & colleagues had subjects play a game where the ability to win money in a given turn of the game was predicted by a visual cue which was presented too briefly to be consciously perceived (see instructions below2). The results of this study suggest that humans are reliably able to subconsciously learn the rewarding value of these visual cues. Importantly, in a control experiment, the researchers demonstrated that the subjects were unable to discriminate between the stimuli without the monetary reward/punishment scheme.

Given the abilities of humans (sketched above) to relegate processing to the subconscious, this finding isn’t that surprising. However, this paper demonstrates the importance of feedback (reward or punishment) for instructing the subconscious. Furthermore, the fact that something as arbitrary as the conscious perception of promised financial reward can serve as the feed-back signal suggests a fundamental role for this type of learning that isn’t limited to certain acts, but might underlie the learning abilities of humans in general.

References/Notes:
1. Pessiglione M, Petrovic P, Daunizeau J, Palminteri S, Dolan RJ, Frith CD. Subliminal instrumental conditioning demonstrated in the human brain. Neuron, 59(4): 561-567, 2008.

2. “The aim of the game is to win money, by guessing the outcome of a button press.

At the beginning of each trial you must orient your gaze towards the central cross and pay attention to the
masked cue. You will not be able to perceive the cue which is hidden behind the mask.

When the interrogation dot appears you have 3 seconds to make your choice between
– holding the button down
– leaving the button up
If you change your mind you can still release or press the button until the 3 seconds have elapsed.

‘GO!’ will be written in yellow if, at the end of the 3 seconds delay, the button is being pressed.

Then we will display the outcome of your choice. Not pressing the button is safe: you will always get a
neutral outcome (£0). Pressing the button is of interest but risky: you can equally win £1, get nil (£0) or
lose £1. This depends on which cue was hidden behind the mask.

There is no logical rule to find in this game. If you never press the button, or if you press it every trial,
your overall payoff will be nil. To win money you must guess if the ongoing trial is a winning or a losing
trial. Your choices should be improved trial after trial by your unconscious emotional reactions. Just
follow your gut feelings and you will win, and avoid losing, a lot of pounds! “

On STDP from Behavior

Figure 6 from ref. 1

I’ve written several times about Spike-Timing-Dependent-Plasticity (STDP), one method by which the individual neurons in the mammalian brain learn; changing their responses to the signals sent from other neurons.

It is believed that STDP is a major route of such learning, both during development and in the adult animal; for instance potentially underlying the associative conditioning famously demonstrated by Pavlov. Indeed, it is just this sort of patterned external sensory stimulus (bell then food) that represents a candidate for learning through STDP. However, connecting the presence of structured environmental variables and underlying brain changes has proven a difficult experimental challenge.

A recent piece of research has achieved just such a feat in the optic tectum of a non-mammal, the developing frog Xenopus laevis1.

I found the figure above to be the most intriguing result from the paper sumarrizing these experiments, published in Nature Neuroscience. The image represents the finding that if the tadpoles are exposed to repetitive flashes of light with a specific time difference between them (top), the neurons in their optic tecta respond by adjusting the latency (time from stimulus to response) of their spike-reactions to single, isolated flashes (middle: neural activity, bottom: histograms of spike latencies).

I am encouraged by this work because it bridges what is currently a rather formidable gap. That between understanding the brain at the level of single neurons and the behavior of an animal as a whole.

References:

1. Pratt, KG, Dong, W & Aizenman, CD (2008) Development and spike timing–dependent plasticity of recurrent excitation in the Xenopus optic tectum Nature Neuroscience 11, 467-475 | doi:10.1038/nn2076

On Learning and Time-Scales

Moo Ming Poo, PhD

On Thursday (2/28/08), I attended a lecture, part of the Neurological Institute at Columbia University’s continuing Seminar series. The talk, titled Activity-Induced Modifications of Neural Circuits was given by Moo Ming Poo, perhaps the most active researcher in this fascinating sub-field, from UC Berkeley. The lecture hall, which has seats for perhaps 80 that are never filled at these events, was packed. Standing room was all that was available, with people spilling into the side aisles and a few intrepid souls (the principle investigator in the lab I work in amongst them) seating themselves in the center aisle as well. Eric R. Kandel, James H. Schwartz, and Thomas M. Jessel (the authors of the most widely used undergraduate and graduate textbook on Neuroscience and all professors at Columbia) were in attendance, as well as countless other researchers and graduate students like myself.

The talk was probably so well attended in part because of Dr. Poo’s notoriety, but also because his work is of some universal intrigue, having relevance in all brain areas and a diverse set research programs.

The title of the seminar refers to the property that individual neurons display in changing the strength of the connections (synapses) between each other in a way that depends on their relative activities. Specifically, if neuron A sends a connection to neuron B, the latter cell needs a way to update the importance of A’s input. Neurons communicate by sending aroundspikes, large pulses of voltage, and it is generally not the case (in the mammalian brain) that a single presynaptic neuron (A) can cause a postsynaptic neuron (B) to fire (spike), rather some several hundred or thousand presynaptic neurons must spike at almost the same time to cause a postsynaptic neuron to fire. I say this because it makes clear the subtle specificity that is needed in synaptic modification, to pick out which of these many inputs have useful information for the cell.

The sensible mechanism, called spike-timing dependent plasticity (STDP), works by up-weighting or strengthening synapses from presynaptic neurons that spiked a short time (20 milliseconds) before the postsynaptic neuron, and down-weighting those in which the presynaptic spike came during a short period after the postsynaptic spike. The large-scale analogy (mentioned by Dr. Poo in his talk) is that of Pavlov’s dog: the canine learned to associate the bell with the meat when the sound preceded the reward, but not when the order was reversed.

In his excellent lecture, Dr. Poo presented some convincing results concerning the molecular mechanisms that might be at work on a microscopic level to achieve this effect, but he concluded with a different and stimulating point.

He presented data from experiments that had been conducted by a post-doctoral researcher in his lab, German Sumbre, using Zebrafish. The results indicated that even very young members of this species are able to learn a predicable series of 60 or so light pulses, delivered every 0.5, 1 or 4 seconds, as indicated by their immaculately timed execution of an escape response called a tail-flick for two or so extra intervals beyond the conclusion of series, right when the pulses would have arrived.

That the fish were able to do this is not incredibly surprising, many animals display this type of predictive behavior. However, it is entirely mysterious how this might be happening at a neuronal level. Dr. Poo has made magnificent progress in understanding how learning proceeds on very short time-scales (10s – 100s of milliseconds), but it is still quite unclear how learning of longer-period phenomena might be achieved by the nervous system. In fact, Dr. Poo appealed to the audience, saying: “If anybody has any ideas how this might be studied, I am anxious to hear them.”

Mirroring the urgency apparent in Dr. Poo’s request, there was a paper published recently on this very topic showing that amoeba are capable of just this sort of learning of intervals1. No doubt this will be a hot topic of research in the near future. Further, this sort of example shows us both how far we’ve come in understanding brains, and the chasms yet to be bridged in moving forward.

References

1. Saigusa T, Tero A, Nakagaki T, Kuramoto Y. (2008) Amoebae anticipate periodic events. Phys Rev Lett. 100(1):018101

On Making Faces

How do babies learn to make faces? With arm or leg movements, it seems plausible that, as William James suggested, one might gain insight from simply associating observed appendage position with concurrent muscle activation patterns1. However, in contrast to an attempted stirring of limbs, the infant cannot see what results from the activation of his facial muscles. Thus, the learning mechanism cannot rely on sensory confirmation that the indented action was successfull. That new humans very rapidly learn to express their emotional state through a smile, frown or furrowing of brow hints that there is some implicit path to the acquisition of this skill. It has been suggested by some that the Mirror Neuron (MN) system constitutes this road, or at least a map of it2.

In the adult, MNs are cells that respond when an animal performs some act – picking up a piece of fruit, say – and when it observes another individual doing the same thing, even if the observed actor is a member of another species or stranger, a robot. Beyond this, their activation potentiates the pathways that would be involved in the execution of such a movement by the observer: producing measurable sub-threshold responses in the muscles involved. The suggestion that this system is involved in motor learning and imitation from birth amounts to the assumption that MNs are prenatally wired to function as they do in the mature brain. This is a very attractive idea as it removes the need for some external form of reinforcement – like a visual confirmation of the completed movement – to inform the motor-learning process. Instead, the responsibility for being both carrot and stick is shifted internally, to the MNs. The proposition is that the genetically defined circuitry imbues the MNs with a “knowledge” of the pattern of muscle activity associated with both an observed or executed behavior.

This, unsurprisingly, presents a further question: if the MN system is present from birth and possesses such information as described above, why do infants need to learn how to move at all? This is where we must tread a bit into the realm of informed speculation. First, the MN system cannot know how to execute every movement possible: for instance it certainly cannot know at birth the set of motor commands associated with performing some complex gymnastic move, say a double backflip. If the MN is an artifact of evolution, then it is likely that there is a continuum of innate interpretability, from simple acts like smiles that are well known to the MN system to more rare or contemporary behaviors like figure skating or fixing a bicycle. Thus, using the MN system as a template, of sorts, can only be effective for behaviors on the oft-encountered end of the spectrum. Second, since babies sadly do not leap from the the womb as masters of muscular control, the wiring of the MN system must itself develop at a pace commensurate with the time-course of an individual’s behavioral procurement.

How is it then, that this internal electro-cultivation proceeds in lock-step with the infant’s newfound agility? It has been well documented that humans lose half the total number of neurons in their central nervous system by the time they’ve reached six months of age. This pruning is a synaptic refinement process, also termed neural darwinism4. It is quite beneficial to the newborn animal, since his neural connectivity is far too manifest, too spatially noisy, and must be cleaned up. This happens in all areas of cerebral cortex. For example, in the visual system, molecular cues guide the axons of nearby retinal ganglion cells to adjacent targets in the thalamus while the animal is still in the womb in a gross way, but it is the activity arising from visual stimulation which pares down the connections to the state we see them in the adult. the exquisite spatial precision of connections between the retina and thalamus cells in the retina are connected to cells in the thalamus with exquisite spatial precision because of visual experience. It was Hebb who pointed out that cells that “fire together wire together.5” This means that if two retinal cells fire at the same time, they will tend to be connected to the same post-synaptic target. That is not to say that any two cells that fire at the same time anywhere in the brain will inevitably be connected to each-other, but rather that in deciding which of the molecularly defined crude connections to keep, a post synaptic cell will retain those which tend to fire at the same time in response to stimulation. The stimuli that cause the retinal cells to fire are not simply sets of independently fluctuating pixels; rather they are full of spatial correlations. If a single vertical line passed across your field of vision, a single line of cells in the retina would be stimulated at once as the line moved by. It is almost never the case that two abutting photoreceptors see completely uncorrelated (in time) patterns of illumination. In this way, the spatial relationships in the image translate into spatial relationships in the connections in the brain.

The goings-on I’ve outlined above might generally be termed activity dependent synaptic refinement (ADSR). What I’m hypothesizing is that, as with vision, some form of ADSR is at work in those cortical-areas involved in learning how to move: that the very act of generating motor output leads to more stereotyped action through application of the Hebbian “fire together wire together” motif. The commonality between vision and movement, and indeed the unifying principle behind ADSR, is that the systems are exploiting the presence of underlying statistical content. While the MN system is biasing motor output towards certain configurations, the motor cortex is learning about the possible relationships between muscle tensions, lengths and contractile velocities. It is thought that the brain might use such a mechanism universally as an attempt at maximizing efficiency. For example: if you always listen to symphonic classical music, you might set your stereo to boost bass & treble, but if you’re more the piano concerto type, you’d want a touch more mid-range; then again, if your tastes are more varied, it might make sense to emphasize all three frequency bands equally so as not to aurally marginalize any particular genre. Another for instance more relevant to motor output: when you drive in the city, you rarely ever get out of 2nd gear, but on the highway, you’re almost always in the upper gears. In the same sense, the brain attempts to use the Hebbian rule to optimize its (sensory) inputs and (motor) outputs to the sensory stimuli impinging on it and the motor programs it generates.

You may have noticed in all this that I’ve skipped over one important point: how (genetic wiring notwithstanding), do mirror neurons extract the information about a movement being performed by another agent? It is not at all understood how this occurs in the adult, so how it could be happening in babies is even more mysterious, especially in light of the messiness of infant brains that I’ve spoken of. It simply must be the case that visual stimuli are translated into data concerning the movement of bodies. It is possible that specialized structures for recognizing arms and hands, faces and feet, become sophisticated very early on, but nothing like this has been observed to-date. The needed course of research is clear, but developing experiments to elucidate what might be at work is not. We can only wait and watch from the wings while the scientific players act, and perhaps deliver soto voce direction from time to time.

References

1. James, W. The Principles of Psychology Vol. 1. Henry Holt & Company (1890)
2. Lepage JF, Théoret H. (2007) The mirror neuron system: grasping others’ actions from birth? Dev Sci. 10(5):513-23.
3. Rizzolatti, G., & Craighero, L., (2004) The Mirror Neuron System. Annu. Rev. Neurosci. 27, 169-192.
4. Hebb, D.O. The Organization of Behavior. John H. Wiley & Sons (1967)
5. Edelman, G.M. Neural Darwinism. Oxford Paperbacks (1990)