Comments on: Amplification using recurrent connectivity http://neurodudes.com/2005/09/08/amplification-using-recurrent-connectivity/ at the intersection of neuroscience and AI. Sun, 07 Sep 2008 02:01:10 +0000 http://wordpress.org/?v=2.6 By: Viren http://neurodudes.com/2005/09/08/amplification-using-recurrent-connectivity/#comment-850 Viren Fri, 09 Sep 2005 06:20:12 +0000 http://neurodudes.com/2005/09/08/amplification-using-recurrent-connectivity/#comment-850 If you are interested in this type of thing, I highly recommend looking into Sebastian Seung's neural network course lecture notes & videos (http://seunglab.mit.edu/courses/9.641/lectures/index.html). The first half of the course takes a very EE-inspired approach to analyzing neural network dynamics in terms of concepts such as amplification, attenuation, and integration. A short summary of these results can be found in: http://hebb.mit.edu/people/seung/papers/hbtnn.pdf The theory of "permitted and forbidden sets" leads to more sophisticated analysis of recurrent networks. For example, one can use the eigenvalues of the synaptic weight matrix to show that certain sets of neurons in a network are "forbidden" from ever being activated together while others are "permitted" to be active together (at stable points of the network activity). This makes it simple to analytically relate synaptic structure to computational function (with relevant examples from associative memory and constraint satisfacation studied in the course notes). Also, these networks (provably) operate within a dynamical regime known as "hybrid computation" that exhibits both digital and analog qualities conjectured to be important properties of biological networks. If you are interested in this type of thing, I highly recommend looking into Sebastian Seung’s neural network course lecture notes & videos (http://seunglab.mit.edu/courses/9.641/lectures/index.html). The first half of the course takes a very EE-inspired approach to analyzing neural network dynamics in terms of concepts such as amplification, attenuation, and integration. A short summary of these results can be found in: http://hebb.mit.edu/people/seung/papers/hbtnn.pdf

The theory of “permitted and forbidden sets” leads to more sophisticated analysis of recurrent networks. For example, one can use the eigenvalues of the synaptic weight matrix to show that certain sets of neurons in a network are “forbidden” from ever being activated together while others are “permitted” to be active together (at stable points of the network activity). This makes it simple to analytically relate synaptic structure to computational function (with relevant examples from associative memory and constraint satisfacation studied in the course notes). Also, these networks (provably) operate within a dynamical regime known as “hybrid computation” that exhibits both digital and analog qualities conjectured to be important properties of biological networks.

]]>