neurodudes

The October 6th issue of Science is a special issue devoted to computational neuroscience. From the introduction to the special issue:

Computational neuroscience is now a mature field of research. In areas ranging from molecules to the highest brain functions, scientists use mathematical models and computer simulations to study and predict the behavior of the nervous system. Simulations are essential because the present experimental systems are too complex to allow collection of all the data. Modeling has become so powerful these days that there is no longer a one-way flow of scientific information. There is considerable intellectual exchange between modelers and experimentalists. The results produced in the simulation lab often lead to testable predictions and thus challenge other researchers to design new experiments or reanalyze their data as they try to confirm or falsify the hypotheses put forward. For this issue of Science, we invited leading computational neuroscientists, each of whom works at a different organizational level, to review the latest attempts of mathematical and computational modeling and to give us an outlook on what the future might hold in store.

Of particular interest is a review article by Randall O’Reilly on biologically based computational models. He focuses on models of the pre-frontal cortex.

This past May, the Almaden Research center, part of IBM research, invited some provocative speakers on the topic of “Cognitive Computing” to come and speak. Since IBM recently invested a lot of money into understanding the brain with the Blue Brain project, it seems like this meeting was a way to figure out the next step along this path.

Powerpoint presentations and videos of the event are available online.

From the synopsis:

The 2006 Almaden Institute will focus on the theme of “Cognitive Computing” and will examine scientific and technological issues around the quest to understand how the human brain works. We will examine approaches to understanding cognition that unify neurological, biological, psychological, mathematical, computational, and information-theoretic insights. We focus on the search for global, top-down theories of cognition that are consistent with known bottom-up, neurobiological facts and serve to explain a broad range of observed cognitive phenomena. The ultimate goal is to understand how and when can we mechanize cognition.

Confirmed speakers include Toby Berger (Cornell), Gerald Edelman (The Neurosciences Institute), Joaquin Fuster (UCLA), Jeff Hawkins (Palm/Numenta), Robert Hecht-Nielsen (UCSD), Christof Koch (CalTech), Henry Markram (EPFL/BlueBrain), V. S. Ramachandran (UCSD), John Searle (UC Berkeley) and Leslie Valiant (Harvard). Confirmed panelists include: James Albus (NIST), Theodore Berger (USC), Kwabena Boahen (Stanford), Ralph Linsker (IBM), and Jerry Swartz (The Swartz Foundation).

In an impressive integrative effort, a new article in this month’s issue of Neuron describes a robust object and face classification model that is consistent with both behavioral and fMRI experiments.

From a preview of the article:

“A central theme that has emerged in research on face perception therefore is whether or not faces are “special” such that the cognitive and neural mechanisms that underlie their processing are different from those underlying the processing of other visual objects. [...] In this issue of Neuron, Jiang et al. (2006) provide a compelling array of evidence supporting the idea that the processing of faces and objects do not rely on qualitatively different mechanisms. In a series of experiments, Jiang et al. present and integrate findings from neural modeling, behavior, and fMRI, showing that face classification, similarly to object classification, can be achieved by a simple-to-complex architecture, based on hierarchical shape detectors. Furthermore, variations of this model can account for both configural and feature-based processing without qualitative modification of the model’s structure.”

The Riesenhuber lab, from which this work comes, has been working on object recognition in an integrative way. The lab is particularly “at the intersection of neuroscience and AI”.

What gives humans the unique ability to construct novel sentences from the building blocks of language? A recent article in Behavioral and Brain Sciences proposes a “neural blackboard architecture” is capable of just this.

From the article (doi: 10.1017/S0140525X06009022):

“This paper aims to show that neural “blackboard” architectures can provide an adequate theoretical basis for a neural instantiation of combinatorial cognitive structures. [...] We also discuss the similarities between the neural blackboard architecture of sentence structure and neural blackboard architectures of combinatorial structures in visual cognition and visual working memory [...]”

As with all main articles in Behavioral and Brain Sciences, this one is followed by extensive comment and criticism from colleagues, and finally a reply by the authors. This provides a very deep look at the article and the issues surrounding it.

An older, but freely available, version of the article is available here.

A recent issue of J Neurosci has a series of mini-reviews on how oscillations play a role in network computations. Two of the reviews are by Sejnowski and collaborators. I haven’t read them yet but I thought I’d post a link here.

Henry Markram, the director of the IBM-sponsored Blue Brain Project has written an article in the latest issue of Nature Reviews: Neuroscience that provides the most technical details about the project to date.

From the article:

“The three-dimensional neurons are then imported into BlueBuilder, a circuit builder that loads neurons into their layers according to a ‘recipe’ of neuron numbers and proportions. A collision detection algorithm is run to determine the structural positioning of all axo-dendritic touches, and neurons are jittered and spun until the structural touches match experimentally derived statistics. [...] Probabilities of connectivity between different types of neuron are used to determine which neurons are connected, and all axo-dendritic touches are converted into synaptic connections. The manner in which the axons map onto the dendrites between specific anatomical classes and the distribution of synapses received by a class of neurons are used to verify and fine-tune the biological accuracy of the synaptic mapping between neurons. It is therefore possible to place 10–50 million synapses in accurate three-dimensional space, distributed on the detailed three-dimensional morphology of each neuron.”

–Stephen

news from the future:

Eugene M. Izhikevich. Polychronization: Computation with Spikes. Neural Computation, Vol. 18, No. 2. (February 2006), pp. 245-282.

This readable, highly recommended paper is about a concept that Izhikevich calls “polychronization”. When a bunch of neurons tend to fire at the same time, you say that group is “synchronized”. But what do you call it when a group of neurons tends to participate in a consistent spatiotemporal firing pattern? Izhikevich would say that such a group of neurons is “polychronized”. Note that synchronization is just a special case of polychronization.

For example, maybe you notice that you often observe the sequence {neuron A fires, then 10ms later neuron B fires, then 14 ms later neuron C fires} in a network. These neurons are not synchronized (they are firing at different times), and the timing of the start of the pattern may or may not be timelocked to anything else (stimulus onset, gamma rhythm, etc), but what is important is that they are firing with a consistent pattern of timing within themselves.

(One complication is that the pattern may not repeat exactly; in the above example, suppose neuron B fails to fire 20% of the time. Just as we say that 3 neurons are mostly synchronized even if one of the neurons fails to fire with the others every single time, we will say that a group is polychronized even if the pattern is inexact [1])

The paper proposes a mechanism which could cause these patterns, and also a mechanism for other neurons to detect them. Both mechanisms are based on axonal conduction delay. As a warm-up, consider how we could get synchronization. Suppose that neurons A, B, and C all synapse onto both neuron D and neuron E, and that the axonal conduction delay is constant (say, 5ms). Now suppose that neurons A, B, and C all fire at the same time. If the synaptic weights are sufficently strong, this will cause D and E to fire at the same time. So, we have synchronization between D and E.

Next, suppose that there are longer delays between A,B,C and neuron E than between A,B,C, and D; it still takes 5ms for a spike to get from A,B, or C to D, but that now it takes 10ms for a spike to get from A,B, or C to E. Now if A,B, and C fire at the same time, it causes a pattern: first D fires, then E fires 5 ms later. This is how axonal conduction delays can cause these patterns.

Now, detection. In the previous two examples, D and E acted as synchrony detectors; they “detected” when A, B, and C fired together. Now what if we add 3ms to the time it takes for a spike to get from neuron C to either D or E? Here are all our axonal conduction delays now:

A->D: 5
A->E: 10
B->D: 5
B->E: 10
C->D: 8
C->E: 13

Now how should A,B,C fire in order to excite D and E? Well, C needs to fire 3ms before A and B if we want D and E to receive 3 spikes at once. So, now D and E are detecting not synchrony, but rather polychrony; they are detecting the pattern {C fires; then 3 ms later, A and B fire}.

Furthermore, D and E’s response to their pattern is not a synchronized burst, but rather it is a different firing pattern; {D fires; then 5 ms later, E fires}.

The first contribution of this paper is in providing us with a language to describe this sort of computational system. In the above example, we might notice that following pattern recurring in network activity: {C fires; then 3ms later, A and B fire; then 5ms later, D fires; then 5ms later, E fires}. The neurons involved in this pattern are termed a polychronous group [1] . Note that a single neuron might participate in multiple polychronous groups; for instance, if the above example is part of a larger network, perhaps there is another firing pattern involving neuron F, G, and C.

The second contribution of this paper is providing a simple 1000-neuron model which exhibits this sort of behavior [2]. The model itself is one page of Matlab code, and is based on the spiking neuron model in (Izhikevich, 2003). The model uses STDP plasticity to update synaptic weights as the model runs, and STDP is key to the formation of the polychronous groups [3].

The third contribution is the analysis of this model. The model displays different network states including slow and fast (”delta” and “gamma”) rhythms. The emergence of polychronous groups is robust to changes in some of the model parameters. Polychronous groups appear and disappear and change over time. Application of an external stimulus can cause a “response” consisting of the probabilistic activation of a certain subset of polychronous groups.

The most important result of the paper in the view of the authors is that the number of polychronous groups exceeds the number of neurons in the network (remember, each neuron may be part of multiple groups). This is important because if the real “logic elements” in the computation are the polychronous groups, not the individual neurons, then the memory capacity or computing capacity of the network may be larger than otherwise expected.

The forth contribution is an interesting discussion of various aspects of how the brain might work if its computation is indeed based on polychronous groups.

FOOTNOTES

[1] Actually, technically, Izhikevich defines a polychronous group based whether the group of neurons has the POTENTIAL to fire in such a pattern, based upon its anatomical connectivity. So, technically, “polychronous group” is an anatomical property, not a functional one.

[2] Although I guess Izhikevich has published similar models before so you might say that “contribution” belongs to his previous papers. Whatever.

[3] Which is analyzed further in (Izhikevich, 2004). Although I don’t think Izhikevich has conclusively shown that STDP is the ONLY kind of plasticity that could cause this.

This post has much the same content as this NeuroWiki page; you may wish to read and comment on it there.

I’ve only skimmed this interesting article, so beware that I may not correctly understand it.

This article proposes that recurrent excitation in cortex leads to amplification, and analyzes this using the mathematics of basic amplifiers (taught in introductory electrical engineering courses; i.e. open-loop gain and closed loop gain).

They construct a simulation based on this principal that agrees with some electrophysiological and pharmacological results from neurobiology experiments in layer IV of V1.

At the end, they conjecture that a sensory network could use this principal for noise reduction and possibly pattern recognition.

Douglas, Rodney J.; Koch, Christof; Mahowald, Misha; Martin, Kevan A. C.; Suarez, Humbert H. Recurrent Excitation in Neocortical Circuits. Science, Volume 269, Issue 5226, pp. 981-985.

For those with theoretical interests with respect to machine learning flavored AI, the ML Theory blog run by John Langford is highly recommended. Though recently started, Langford and others have so far been doing an excellent job of commenting on both the science and culture of theoretical learning research.

Neuroimaging data of different brain areas fit to a Rescorla-Wagner model show that different cortical areas integrate stimulus changes over different time intervals. The result itself probably isn’t that shocking but I liked the nice combination of theory and experiment.

From the July 21 Neuron:

Formal Learning Theory Dissociates Brain Regions with Different Temporal Integration

Jan Gläscher and Christian Büchel

Learning can be characterized as the extraction of reliable predictions about stimulus occurrences from past experience. In two experiments, we investigated the interval of temporal integration of previous learning trials in different brain regions using implicit and explicit Pavlovian fear conditioning with a dynamically changing reinforcement regime in an experimental setting. With formal learning theory (the Rescorla-Wagner model), temporal integration is characterized by the learning rate. Using fMRI and this theoretical framework, we are able to distinguish between learning-related brain regions that show long temporal integration (e.g., amygdala) and higher perceptual regions that integrate only over a short period of time (e.g., fusiform face area, parahippocampal place area). This approach allows for the investigation of learning-related changes in brain activation, as it can dissociate brain areas that differ with respect to their integration of past learning experiences by either computing long-term outcome predictions or instantaneous reinforcement expectancies.

How does this relate to Hawkins’s idea that all cortex implements the same underlying “algorithm”? Is the integration time constant (or, in RW terms, the learning rate) tuned differently by different inputs?