neurodudes

I’ve only skimmed this article by Ranganathan, but I find it notable because of the discussion of memory-oriented computing, in which processors are colocated with storage (he uses the word “nanostores”, which additionally implies that the memory is nonvolatile). One of the most important distinctions between neural architecture and present-day computing architecture is that brains appear to be built out of computing elements that do both processing and memory storage, whereas present-day computers have separate memory and CPU components (this separation is a key feature of what is called the “von Neumann” architecture).

This separation means that computation is often rate-limited by the speed at which information can be transferred between memory and the CPU, referred to in John Backus’s Turing Award lecture as “the von Neumann bottleneck”. In Danny Hillis’s book “The Connection Machine” (which I highly recommend), he argues that the von Neumann architecture additionally unnecessarily slows down computation because most of the silicon in a computer is sitting there unused most of the time when it’s being used to store memories which are not currently being accessed (Hillis proposed solution was massively parallel memory-oriented computing).

In addition to Hillis’s argument that the von Neumann design is temporally inefficient, this NYTimes commentary on Ranganathan’s article argues that it is energy inefficient, citing a panel that found that the energy cost of moving data between memory and processors is more than 10x the energy cost of the processing itself (and possibly more than 100x). In other words, massively parallel memory-oriented computing, which seems to be how the brain works, may be both faster and more energy-efficient that von Neumann computing (what’s the catch? You have to write massively parallelizable algorithms to run on it). The energy-efficiency part of this isn’t too surprising, as evolution had a lot of selection pressure to optimize for low energy use. It’s neat though.

Parthasarathy Ranganathan. From Microprocessors to Nanostores: Rethinking Data-Centric Systems. IEEE Computer January 2011, p. 39-48.

NYTimes summary: Remapping Computer Circuitry to Avert Impending Bottlenecks – NYTimes.com.

This entry was posted on Wednesday, March 2nd, 2011 at 1:26 am by Bayle Shanks and is filed under Distributed/Parallel Computation, Interdisciplinary concepts. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.