neurodudes

(pun intended). I am embarrassed to say that earlier today I remarked to a colleague that dopamine only encodes unexpected reward, not unexpected lack of reward. This is (afaik) incorrect. It has a baseline level of firing that goes down when there is an unexpected lack of reward (see fig 1 in Wolfram Schultz, Peter Dayan, P. Read Montague. A Neural Substrate of Prediction and Reward)

However, because it can only go down so far, the negative signal is clipped, which might have consequences (see Yael Niv, Michael O Duff, Peter Dayan. Dopamine, uncertainty and TD learning).

The previous article mentions that some other people think that maybe dopamine is tracking uncertainty as well as reward. This one talks about a theory that acetylcholine is related to expected uncertainty, and norepinephrine is related to unexpected uncertainty:
Angela Yu, Peter Dayan. Expected and Unexpected Uncertainty: ACh and NE in the Neocortex (huh, all those papers had Peter Dayan as one of the authors) (btw I haven’t read all of the papers I’m posting here)

Since we’re on the subject of temporal difference learning, I’ll mention that in my opinion temporal difference learning may be a model of how futures/speculators in financial markets are supposed to propagate future price changes back in time to the present (if you think of the market as a cognitive system). I haven’t formalized this idea yet, though.