This separation means that computation is often rate-limited by the speed at which information can be transferred between memory and the CPU, referred to in John Backus’s Turing Award lecture as “the von Neumann bottleneck”. In Danny Hillis’s book “The Connection Machine” (which I highly recommend), he argues that the von Neumann architecture additionally unnecessarily slows down computation because most of the silicon in a computer is sitting there unused most of the time when it’s being used to store memories which are not currently being accessed (Hillis proposed solution was massively parallel memory-oriented computing).

In addition to Hillis’s argument that the von Neumann design is temporally inefficient, this NYTimes commentary on Ranganathan’s article argues that it is energy inefficient, citing a panel that found that the energy cost of moving data between memory and processors is more than 10x the energy cost of the processing itself (and possibly more than 100x). In other words, massively parallel memory-oriented computing, which seems to be how the brain works, may be both faster and more energy-efficient that von Neumann computing (what’s the catch? You have to write massively parallelizable algorithms to run on it). The energy-efficiency part of this isn’t too surprising, as evolution had a lot of selection pressure to optimize for low energy use. It’s neat though.

Parthasarathy Ranganathan. From Microprocessors to Nanostores: Rethinking Data-Centric Systems. IEEE Computer January 2011, p. 39-48.

NYTimes summary: Remapping Computer Circuitry to Avert Impending Bottlenecks – NYTimes.com.

A few days later, Henry Markram, leader of the Blue Brain Project at EPFL, sent off an e-mail to IBM CTO Bernard Meyerson harshly criticizing the IBM press release, and cc’ed several reporters. This brought a spate of shock media into the usually placid arena of computational neuroscience reporting, with headlines such as “IBM’s cat-brain sim a ‘scam,’ says Swiss boffin: Neuroscientist hairs on end”, and “Meow! IBM cat brain simulation dissed as ‘hoax’ by rival scientist”.  One reporter chose to highlight the rivalry as cat versus rat, using the different animal model choice of the two researchers as a theme.  Since then, additional criticisms from Markram have appeared online.

Find out more after the jump.

In the aftermath, IBM has stood behind the announcement, citing for Network World their team’s involvement with “Stanford University, University of Wisconsin-Madison, Cornell University, Columbia University Medical Center, University of California-Merced and Lawrence Berkeley National Laboratory” as defense.  Who are the researchers they are standing behind?  According to Modha’s blog, they are:

• Stanford University: Brian A. Wandell (Prof of Psychology, Electrical Engineering), H.-S. Philip Wong (Prof of Electrical Engineering)
• Cornell University: Rajit Manohar (Prof of Electrical Engineering)
• Columbia University Medical Center: Stefano Fusi (Prof of Theoretical Neuroscience)
• University of Wisconsin-Madison: Giulio Tononi (Prof of Psychiatry)
• University of California-Merced: Christopher Kello (Prof of Cognitive Science)

For this neurodude, it is interesting how this disagreement may be symbolic of the gap that still remains between neuroscience and AI.  Markram is a neuroscientist turned technologist, while Modha is a computer engineer who wants to derive technological insight from biological  systems.  They are approaching the ideal of reverse engineering the brain from very different perspectives, and its only natural that they value different milestones.  The IBM team, even with the additional professors on their team, still lacks mainstream neuroscientists to help validate their claims.  That being said, the public realization of this could be a positive thing for both fields.  Although some frustration has resulted from this, this could be a great opportunity for the breakdown of walls between these fields.

In the end though, it does seem like Markram has a point.  The IBM press release clearly went too far.  Whether the angry public e-mail was the best strategic way to make the point remains to be seen.  It will be interesting to see what the next move from the IBM team will look like.

To bring you up to speed, I need to remind you what is going on in the world of experimental neuroscience.

Experimentalists are now able to record the single-cell activities of a whole population of neurons simultaneously. From Briggman, Abarbanel, Kristan (2006):

By using multi-electrode arrays or optical imaging, investigators can now record from many individual neurons in various parts of nervous systems simultaneously while an animal performs sensory, motor or cognitive tasks. Given the large multidimensional datasets that are now routinely generated, it is often not obvious how to find meaningful results within the data.

This paper goes on to provide a nice overview on mathematical methods that researchers are using to grapple with the challenge of understanding the dynamics of the neural systems they are recording from. They make the case that conceptual progress needs to be made on the interpretation of the data these results yield. How can we understand what computations these neurons are collectively performing?

(Incidentally, this topic is being explored in a conference happening this week at the Los Alamos National Laboratory, which, according to one of the conference session chairs, is intended to help shape future directions for the lab. Hopefully there will be webcasts from this conference.)

Theorists have worried about what neurons are doing in local populations for some time. Investigations have given rise to all kinds of models of such activity, notably those of Wilson & Cowan (1972), and Hopfield (1982), which viewed a network of neurons as having “attractor states”, invoking the language, for Hopfield, and the mathematics, for Wilson & Cowan, of dynamical systems theory.

In 2002, Maass, Natschlager and Markram proposed a theory of computation for localized populations of neurons that did not require that the population settle into a stable attractor in order to perform computations. The idea is that local populations of interconnected neurons of the right kinds are capable of discriminating temporal input patterns in general, and that this behavior is governed by the network dynamics of those populations. They showed that neuronal circuits can be constructed to create a generalizable computational architecture for continuous analog systems, known as liquid state machines. In addition, these circuit models are some of the first to incorporate short-term synaptic plasticity in a dynamical population model.

This idea, elaborated by Maass & Markram in a book chapter entitled Theory of the computational function of microcircuit dynamics, circa 2005, proposes that the liquid state machine architecture is capable of universal computation, just as Turing machines are.

On the heels of these postulates, two recent papers make a serious effort to combine this theoretical paradigm with experimental data. Haesler & Maass (2007), in Cerebral Cortex, synthesized data from layers of cerebral cortex to build a sophisticated dynamical model and tested to see if it had the kinds of computational properties described by the previous theoretical work (it did). Karmarkar & Buonomano (2007), in Neuron, explored the notion of “clockless computing” by networks of model neurons to understand how neural systems tell time, and used psychophysical experiments to support their theories.

A few concrete things are suggested by these works:

1. Computational models of neuronal networks that take short-term plasticity and other biological details into account can be constructed. They demonstrate relevant computational properties.
2. Testing the computational properties of such networks requires framing experiments in terms of complex analog signal processing.
3. The global dynamical properties of local populations of neurons set the general tone of the computations they perform, while the single cell dynamics shape and refine those computations.

Together, these papers may represent the beginning of a new understanding of the computations that networks of real physiological neurons are capable of. Expectations are high that results from further multi-cellular recordings and from the Blue Brain project will verify and elaborate these ideas. Stay tuned!

HOW CAN TECHNOLOGY EMULATE BIOLOGICAL INTELLIGENCE?

The conference is aimed at researchers and students of computational neuroscience, cognitive science, neural networks, neuromorphic engineering, and artificial intelligence. It includes invited lectures and contributed lectures and posters by experts on the biology and technology of how the brain and other intelligent systems adapt to a changing world. The conference is particularly interested in exploring how the brain and biologically-inspired algorithms and systems in engineering and technology can learn. Single-track oral and poster sessions enable all presented work to be highly visible. Three-hour poster sessions with no conflicting events will be held on two of the conference days. Posters will be up all day, and can also be viewed during breaks in the talk schedule.

ELEVENTH INTERNATIONAL CONFERENCE
ON COGNITIVE AND NEURAL SYSTEMS

May 16 – 19, 2007

Boston University
677 Beacon Street

Boston, Massachusetts 02215 USA

http://www.cns.bu.edu/meetings/

Sponsored by the Boston University

Center for Adaptive Systems and
Department of Cognitive and Neural Systems (http://www.cns.bu.edu/)
with financial support from the National Science Foundation (http://cns.bu.edu/CELEST/)

Kevin Lafferty, a parasitologist, has put forth the idea that a fairly ubiquitous parasite (infecting O(10%) of Americans, and up to 2/3 of people in places like Brazil) is responsible for some of the diversity of human culures (1). The parasite uses common housecats to increase its transmission to the next host in the life cycle, and has a subtle effect on human personality, with some studies claiming that it even causes neuroticism, and even schizophrenia. (One clinical report (2) claims that “subjects with latent toxoplasmosis had higher intelligence [and] lower guilt proneness.” Hmm!)

Anyway, Lafferty noted that toxoplasmosis varies in prevalence from world region to world region, and then tries to draw correlates between these prevalences and local cultures:

“Drivers of the geographical variation in the prevalence of this parasite include the effects of climate on the persistence of infectious stages in soil, the cultural practices of food preparation and cats as pets. Some variation in culture, therefore, may ultimately be related to how climate affects the distribution of T. gondii, though the results only explain a fraction of the variation in two of the four cultural dimensions, suggesting that if T. gondii does influence human culture, it is only one among many factors.”

I wonder how one could test this hypothesis? Look for recent immigrants from one culture to another, who have lower Toxoplasmosis incidence? (Preferably finding populations that go in opposite directions, as a control.) Track culture change vs. migration vs. climate change?

Unlikely, perhaps. But nice that people are still thinking big

– Ed

(1) Lafferty, K
Can the common brain parasite, Toxoplasma gondii, influence human culture?
Proceedings of the Royal Society B: Biological Sciences
doi:10.1098/rspb.2006.3641

Picked up by the popular press here

(2) Flegr J, Havlicek J.
Changes in the personality profile of young women with latent toxoplasmosis.
Folia Parasitol (Praha). 1999;46(1):22-8.

Very few MEA studies make it into Nature, so this definitely got my attention.

Often in neuroscience we are confronted with a small sample measurement of a few neurons from a large population. Although many have assumed, few have actually asked: What are we missing here? What does recording a few neurons really tell you about the entire network?

Using an elegant prep (retina on a MEA viewing defined scenes/stimuli), Segev, Bialek, and students show that statistical physics models that assume pairwise correlations (but disregard any higher order phenomena) perform very well in modeling the data. This indicates a certain redundancy exists in the neural code. The results are also replicated with cultured cortical neurons on a MEA.

Some key ideas from the paper are presented after the jump.

To describe the network as a whole, we need to write down a probability distribution for the 2N binary words corresponding to patterns of spiking and silence in the population. The pairwise correlations tell us something about this distribution, but there are an infinite number of models that are consistent with a given set of pairwise correlations. The difficulty thus is to find a distribution that is consistent only with the measured correlations, and does not implicitly assume the existence of unmeasured higher-order interactions.

…

Therefore, the question of whether pairwise correlations provide an effective description of the system becomes the question of whether the reduction in entropy that comes from these correlations, I(2) = S1 – S2, captures all or most of the multi-information IN.

…

We conclude that although the pairwise correlations are small and the multi-neuron deviations from independence are large, the maximum entropy model consistent with the pairwise correlations captures almost all of the structure in the distribution of responses from the full population of neurons. Thus, the weak pairwise correlations imply strongly correlated states. To understand how this happens, it is useful to look at the mathematical structure of the maximum entropy distribution.

…

In a physical system, the maximum entropy distribution is the Boltzmann distribution, and the behaviour of the system depends on the temperature, T. For the network of neurons, there is no real temperature, but the statistical mechanics of the Ising model predicts that when all pairs of elements interact, increasing the number of elements while fixing the typical strength of interactions is equivalent to lowering the temperature, T, in a physical system of fixed size, N. This mapping predicts that correlations will be even more important in larger groups of neurons.

And of note from the Discussion:

The dominance of pairwise interactions means that learning rules based on pairwise correlations could be sufficient to generate nearly optimal internal models for the distribution of ‘codewords’ in the retinal vocabulary, thus allowing the brain to accurately evaluate new events for their degree of surprise.

The project, dubbed “Blue Brain“, represents a team up between Henry Markram, (who co-authored the chapter on the neocortex in the acclaimed reference The Synaptic Organization of the Brain), and IBM’s Blue Gene super computer.

From the New Scientist article: For over a decade Markram and his colleagues have been building a database of the neural architecture of the neocortex, the largest and most complex part of mammalian brains.

Very thin slices of mouse brain were kept alive under a microscope and probed electrically before being stained to reveal the synaptic, or nerve, connections. “We have the largest database in the world of single neurons that have been recorded and stained,” says Markram.

–Stephen

“Leaves have openings called stomata that open wide to let CO2 in, but close up to prevent precious water vapour from escaping. Plants attempt to regulate their stomata to take in as much CO2 as possible while losing the least amount of water. But they are limited in how well they can do this: leaves are often divided into patches where the stomata are either open or closed, which reduces the efficiency of CO2 uptake.

By studying the distributions of these patches of open and closed stomata in leaves of the cocklebur plant, Peak and colleagues found specific patterns reminiscent of distributed computing. Patches of open or closed stomata sometimes move around a leaf at constant speed, for example.

The statistics of the size of these patches, and of the waiting times between the appearance of successive patches, are the same as those for a model of cellular automata, the researchers say. The individual leaf stomata appear to act like simple computers, responding to what their neighbouring stomata are doing.

The researchers think that transient patchiness may be the price the plant pays for a reasonably efficient and simple way form of computation. It is a sign of the plant ‘thinking’ while it figures out the best solution to the problem of how much to open its stomata.
”

– Philip Ball. Do plants act like computers? Leaves appear to regulate their ‘breathing’ by conducting simple calculations. Nature Science Update, 21 January 2004

Peak, D. A., West, J. D., Messinger, S. M & Mott, K. A. Evidence for complex, collective dynamics and emergent, distributed computation in plants. Proceedings of the National Academy of Sciences USA, 101, 918 – 922, (2004).