However, because it can only go down so far, the negative signal is clipped, which might have consequences (see Yael Niv, Michael O Duff, Peter Dayan. Dopamine, uncertainty and TD learning).

The previous article mentions that some other people think that maybe dopamine is tracking uncertainty as well as reward. This one talks about a theory that acetylcholine is related to expected uncertainty, and norepinephrine is related to unexpected uncertainty:
Angela Yu, Peter Dayan. Expected and Unexpected Uncertainty: ACh and NE in the Neocortex (huh, all those papers had Peter Dayan as one of the authors) (btw I haven’t read all of the papers I’m posting here)

Since we’re on the subject of temporal difference learning, I’ll mention that in my opinion temporal difference learning may be a model of how futures/speculators in financial markets are supposed to propagate future price changes back in time to the present (if you think of the market as a cognitive system). I haven’t formalized this idea yet, though.

A few days later, Henry Markram, leader of the Blue Brain Project at EPFL, sent off an e-mail to IBM CTO Bernard Meyerson harshly criticizing the IBM press release, and cc’ed several reporters. This brought a spate of shock media into the usually placid arena of computational neuroscience reporting, with headlines such as “IBM’s cat-brain sim a ‘scam,’ says Swiss boffin: Neuroscientist hairs on end”, and “Meow! IBM cat brain simulation dissed as ‘hoax’ by rival scientist”.  One reporter chose to highlight the rivalry as cat versus rat, using the different animal model choice of the two researchers as a theme.  Since then, additional criticisms from Markram have appeared online.

Find out more after the jump.

In the aftermath, IBM has stood behind the announcement, citing for Network World their team’s involvement with “Stanford University, University of Wisconsin-Madison, Cornell University, Columbia University Medical Center, University of California-Merced and Lawrence Berkeley National Laboratory” as defense.  Who are the researchers they are standing behind?  According to Modha’s blog, they are:

• Stanford University: Brian A. Wandell (Prof of Psychology, Electrical Engineering), H.-S. Philip Wong (Prof of Electrical Engineering)
• Cornell University: Rajit Manohar (Prof of Electrical Engineering)
• Columbia University Medical Center: Stefano Fusi (Prof of Theoretical Neuroscience)
• University of Wisconsin-Madison: Giulio Tononi (Prof of Psychiatry)
• University of California-Merced: Christopher Kello (Prof of Cognitive Science)

For this neurodude, it is interesting how this disagreement may be symbolic of the gap that still remains between neuroscience and AI.  Markram is a neuroscientist turned technologist, while Modha is a computer engineer who wants to derive technological insight from biological  systems.  They are approaching the ideal of reverse engineering the brain from very different perspectives, and its only natural that they value different milestones.  The IBM team, even with the additional professors on their team, still lacks mainstream neuroscientists to help validate their claims.  That being said, the public realization of this could be a positive thing for both fields.  Although some frustration has resulted from this, this could be a great opportunity for the breakdown of walls between these fields.

In the end though, it does seem like Markram has a point.  The IBM press release clearly went too far.  Whether the angry public e-mail was the best strategic way to make the point remains to be seen.  It will be interesting to see what the next move from the IBM team will look like.

Other training videos from the same trainer are available along with an official website with interesting tips about mouse training. Perhaps highly inbred lab mice are unable to replicate such feats but it is amazing to see in what detail this trainer understands mouse behavior and development:

An absolute necessity for any pet training is to understand the animal’s needs and to know about its generic behaviour, since appropriate animal training is only based on certain natural habits. For mouse agility, this means e.g. their great spatial orientation abilities and spatial memory which is worth bringing to light by relevant trick training. In nature, mice always prefer the familiar (= safe) route to their feeding site, no matter if it’s a long way round. This is also the reason why mice are unbeatable in maze tests – and a mouse agility course is nothing else than a maze without walls!
But many owners forget that if you expect your pet to show some natural habits and abilities, first and foremost the husbandry has to be species-appropriate. If your mice have to live in a small ground level cage, their three-dimensional consciousness and orientation abilities will surely be stunted or never fully develop.

Click on for some of my favorite passages from the Abbott piece.

Abbott uses the following problem of input decoding

Spike counts and neuronal firing rates are positive quantities. This simple fact has important implications for neural coding and neural circuits that provide a framework for thinking about a number of research directions taken over the past 20 years.

to highlight new work in synchrony, dendritic compartments, and balanced excitation-inhibition. This is probably the best part of the whole article. With some simple arithmetic, he motivates and explains solutions to the problem of correlating neural activity with real events.

The successes of circuit models (and principles of circuit models) in primary visual cortex:

We now have plausible mechanisms for how simple and complex cells obtain their basic response characteristics. Although no single consensus abouthow the circuits of primary visual cortex operate has arisen from this body of work, this may simply reflect the fact that multiple mechanisms contribute. In other words, many of these ideas are probably correct in one way or another, and the wealth of ideas in this field should be viewed as a success. Circuit-level modeling is now advancing beyond primary sensory areas (for example, Cadieu etal., 2007) and to the consideration of phenomena such as working memory through sustained activity (Amit and Brunel, 1997,Compte etal., 2000,Seung etal., 2000) and decision making (Wang, 2002,Machens etal., 2005).

And the dangers of an unhealthy obsession with connectomics:

What can we learn from the complete connectome or, indeed, a complete mathematical description of a complex artificial network model?

First, what can’t we learn? It is unlikely, for example, that we could deduce the task that the network was constructed to perform even if we were given the complete equations and connections of the model. If, along with this information, we were told what this task was, it is unlikely that we could figure out how the network performs it. If we somehow managed to make any progress along these lines, the people who constructed the network could probably provide us with another one that performs the same task but has a different connectome. In a similarway, biological systems may operate in a more variable manner than we have suspected, as has been stressed by Eve Marder (Marder etal., 2007). These issues are particularly true of a class of network models known as liquid state or echostate networks (Maass etal., 2002,Jaeger, 2003). In these models, the vast majority of interneuronal connections are not directly related to the task being performed (they are typically chosen randomly and left unchanged), the exceptions being synapses onto the output units of the network. Nevertheless, the tuned values of the synapses onto the output units can only be understood through their relationships to the random synapses. Such systems represent enormous challenges for conventional anatomical and physiological approaches.

The fact that the connectome of an artificial neural network does not typically tell us what the network does or how it does it should not be taken as an indication that this information is useless. Far from it. But we must be willing to be more abstract in our thinking. The important issue for an artificial network is not how it works but how it was constructed, which means what training procedures and modification rules were used to get it to perform a task. Although this information is not provided directly by the connectome, much can be inferred. For example, it is important to know whether the network has a feedforward architecture or has strong feedback loops. Other features of the network layout, whether it has hubs or bottlenecks, how many layers it contains, and its degree of heterogeneity, provide important clues as well. Obtaining a high-resolution connectome in neuroscience will be of great value, but artificial neural networks provide a cautionary tale that reminds us that scientific revolutions tend to render uninteresting as many questions as they answer. We will be fortunate if the connectome project does this for neuroscience, butas we launch ourselves into it we should appreciate that, as artificial neural networks appear to suggest, we may be asking the wrong questions.

Finally a major challenge for the future:

This is where I think the future lies in theoretical investigations of cognitive function. We must learn how to build models that construct hypotheses through their internally generated activity while remaining sensitive to the constraints provided by externally generated sensory evidence.

The paper can be found Here.

Daw et al. find that humans choose between multiple slot machines (with different payoff probabilities) based on expected value (versus just going with the highest probability one most of the time and then randomly choosing another one every so often). Then, with fMRI, they find brain areas correlated with different value predictions.

News & Views (Daeyol Lee)

Cortical substrates for exploratory decisions in humans (Daw, Dayan)

Abstract:

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this ‘exploration-exploitation’ dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects’ choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.

I haven’t gotten a chance to fully digest this article (What is the attenuation phenomena that happens when the taps are delayed?), but it seems like a deep result from a relatively simple haptics experiment. Just thought I’d share it with the crowd.

Also, Happy Birthday to fellow Neurodude Bayle! Congrats, man.

They determine the effect of a tetanus at one electrode in a network on the network. Specifically, they look at how the tetanus potentiates or depresses the ability of a test pulse at another electrode to evoke spike trains at various neurons across the network.

They grew cultures on a MEA for a month. They stimulated each electrode in succession with a test pulse. They recorded the response at all electrodes after each test pulse. They used spike sorting to identify the reponses of individual neurons out of the electrode traces. They found that the network’s response to a given test pulse was reproducable for about 50ms after the test pulse.

Then they applied a strong stimulus (a tetanus) to a single electrode (to make it learn ). After that they re-characterized the network’s responses to test pulses at every site.

They found that some electrode sites became more potent (“potentiated response”) after the tetanus was applied. This means that, when a test pulse was applied to this electrode site, neurons in all areas of the network responded either the same, or more strongly than they had before the tetanus.

Other sites became less potent (“depressed response”) after the tetanus was applied.

Surprisingly, it was very rare for any given electrode site to become better at stimulating some neurons and worse at stimulating others as a result of the tetanus.

What determined which electrode sites became potentiated and which ones became depressed? The tetanus potentiated electrodes which evoked spike trains that tended to contain spikes which were within 40ms of the spike trains evoked by the tetanus electrode, and depressed others. That is, it potentiated sites which evoked patterns similar to the patterns evoked by the tetanus site.

However, the spike trains evoked by both potentiated and depressed neurons became more synchronized with the tetanus electrode after applying the tetanus.

See page 5 of “Distributed processing in cultured neuronal networks” for another review of this work.

See this NeuroWiki page for more details (the strange {{}} over there are because we will soon have footnotes).

Jimbo, Y., Tateno, T., and Robinson, H. P. C.,
Simultaneous Induction of Pathway-Specific Potentiation and Depression in Networks of Cortical Neurons. Biophysical Journal, 1999. 76: p. 670-678.