README

For any questions, comments or bug reports please email mfrank@u.arizona.edu

NOTE: The models are set up to run under PDP++ 3.2a08, and the
 analysis scripts configured to work under Linux.  If you are
 running an older version you may have to download and install the
 more recent version of PDP/leabra from
 ftp://grey.colorado.edu/pub/oreilly/pdp++/. If you get error messages
 when you load the project, please let me know or try to download that
 version (newer PDP++ versions should work too -- but this project has
 not yet been converted to emergent). The below instructions assume
 some basic experience with the software; please refer to the O'Reilly
 & Munakata (2000) textbook for an introduction.

----------------------------------------------------------------------------

Simulating effects of Deep Brain Stimulation of the Subthalamic
Nucleus on Probabilistic Decision Making 

leabra++ Conflict_DBS.proj.gz

This project replicates the basic effects of removing the STN from
processing (ie., a "lesion"), and applying external high frequency
activation to the STN (DBS of the STN) on conflict-induced slowing, as
described in Frank et al (2007), Science and Frank (2006), Neural
Networks. (The same models were also used to simulate Parkinson's
disease and dopamine medication effects on Go and NoGo learning, and
performance in the Weather Prediction task and probabilistic reversal,
as described in Frank et al 2004, Science, and Frank (2005), J Cog
Neurosci. For those simulations, please refer to the README file and
associated projects in BG_DA_Learn.zip, also available on ModelDB or
via email).

The network is trained with two stimuli (A and B), associated with
probabilistic feedback, using the Train_Prob process. After each epoch
of training, the network is tested twice using the Epoch_1 and Epoch_4
processes. In both of these tests the network weights are frozen (they
do not change as a function of choice) so as to assess performance of
the network on test trials without allowing them to learn about these
test trials. In this way we can assess conflict based responding after
various stages of training without contaminating subsequent results.
In Epoch_1, the network is tested with the single cues from the
training environment that are associated with 80 or 60% reward
probability. In Epoch_4, the network is tested with a high conflict
combination of the two cues, each associated with a different response
(one is 80% R1 while the other is 60% R2). This is analogous to the
situation in which healthy individuals show slowed reaction times,
relative to low conflict trials. To increase reliability of the
measure, each of the epochs consists of two identical (low or high
conflict) trials -- in principle, the same network with the same
weights could select different responses in these two trials, and with
different reaction times, simply due to noise in unit activity).

***REACTION TIME MEASURES***

To measure each network's reaction time for the different trial types,
 use the Cycle_1 and Cycle_4 Graphlogs, which monitor the activity
 levels of each layer in the network after every cycle of network
 processing within each trial (where again Cycle_1 refers to low
 conflict and Cycle_4 is high conflict). We will keep track of these
 activity levels and compute BG model reaction times in terms of the
 number of cycles until the output activation exceeds a threshold (see
 below). First, make sure that the network is in the desired condition
 (intact, lesion, DBS, meds, etc.) by selecting a setting on the
 "SelectEdit" control panel.  Clicking "intact Run" will put the
 network into its intact state, to simulate normal network
 functioning.  Checking the box marked "STN lesion" will remove the
 STN from processing altogether.  The "DBS" condition adds external
 high frequency stimulation to the STN, functionally reducing its
 ability to respond adaptively to its cortical inputs (see Supplement
 of Frank et al, 2007). The "PD" button will reduce DA in the SNc
 layer, so that only 2 of the 4 units project DA to the Striatum, and
 limiting DA bursts during positive feedback.  And finally, selecting
 "meds" will increase DA relative to the PD networks and also reduce
 the DA dips during negative feedback. These were the same
 manipulations performed in Frank, 2005 and Frank et al 2004. 

Ater selecting the desired model condition, click on LogFile > Set
 Save File in the the Cycle_1 and Cycle_4 graphlogs and name the new
 .log file (e.g., intact_c1.log). This will save all information
 collected into the corresponding file for later analysis. Set the
 number of networks to train (each with its own set of random initial
 weights) in the Batch_Prob control panel by entering a number in
 "batch.max" (25 or 50 should be sufficient), hit "Reinit" and then
 "Run."

(Note: simulations will run considerably faster if you turn off the
network and graphlog updating displays, by checking off the button
marked Display on the top left corner of the network window).


Once all networks have been run, you can analyze the resulting log
file and determine when each response was made. We have provided a css
script which does this for you (css is the scripting language
associated with pdp). The script, called
"anal_rt_thal_e20.css", analyzes model reaction times, in terms of the
number of network processing cycles (where each cycle consists of a
single update of units membrane potentials and activity states) until
a response is selected by BG circuitry. Because the BG (GPi) is
tonically inhibiting the thalamus, the best measure of when a response
is selected is when a given Thalamic unit becomes disinhibited (no
longer has zero activation; in contrast, cortical units have noisy
activation during the entire settling process so if activation exceeds
some value for a single cycle in cortex that should not necessarily be
taken to indicate that a response was selected). Once the thalamus is
disinhibited and a response selected, a single thalamic unit's
activity rises sharply toward maximal firing rate (1.0). We used an
arbitrary threshold of 50% maximal firing rate for a given Thalmus
unit in order to quantify the reaction time -- but this arbitrary
value does not influence the general pattern of results. 

To use this script, type from a console "anal_rt_thal_e20.css
<logfilename.log>". After churning away for a bit, this will produce a
new file with the extension .gort20 (gort refers to the reaction time
at which a "Go" signal facilitated a response).  This file contains
the reaction time information (number of cycles) for every network
that was run in the batch. Note that the script computes reaction
times only after 20 epochs of training (hence the "e20" in the script
name and gort20 in the output filename), once they have learned the
probabilistic associations. Optionally, one can use the
"anal_rt_thal.css" script (ie without the "e20") to compute RT's for
every trial across training, which produces qualitatively similar
results (but as expected, conflict effects are maximal after
learning).
 
 After a .gort20 file is created, another script can be used to
 average these results across all networks in a batch (assuming you
 have the old unixstat package with "dtsem" installed -- otherwise you
 will have to use your own spreadsheet or other analysis tool for
 theses stats).  This script is called "analrt.sh", and the resulting
 file will have the extension .gort20.dt.  For example, type
 "analrt.sh intact_c1.log.gort20."  The .dt output file will contain
 four columns of information.  The first colum is simply the letters
 "GM," and can be ignored.  The three columns of interest, from left
 to right, are the number of trials for which a response was made (ie
 a thalamus unit's activation surpassed the threshold within the
 maximum of 200 cycles -- after this time we consider this to be a
 non-response), followed by the mean number of cycles until the
 response was selected, and finally, the standard error of the
 mean. Your results should show that lesion and DBS nets are
 considerably faster than the other networks when faced with high
 conflict (_c4 results), and also somewhat faster than their low
 conflict reaction times. This high conflict speeding effect was
 observed in patients on DBS. In the model this occurs due to the
 presence of multiple Go signals and simulated DA increases during
 win/win conditions (SNC unit firing rate levels were increased from
 the normal tonic level of 0.5 to 0.7 during the test phase to
 simulate the presence of a rewarding stimulus. As described in the
 supplement of Frank et al (2007), if DA levels are not increased and
 remain at the tonic level, STN dysfunction still reduces
 conflict-induced slowing but does not actually cause high-conflict
 speeding, as we observed for lose/lose high conflict choices in DBS
 patients).

***ACCURACY MEASURES***

 To measure the network's accuracy, again make sure that the network
 is in the desired condition by selecting "intact," "DBS," etc., on
 the control panel as described above. After selecting the desired
 condition, find the Epoch_1 and Epoch_4 Graphlogs.  Select "Save Set
 File" and name the new .log file (e.g, "intact_e1.log" (low conflict)
 and "intact_e4.log" (high conflict)). Then, in the Batch_prob
 process, set the number of networks to run and hit Reinit and Run.
 After the model has finished running the batch, accuracy results can
 be analyzed by using the following scripts: analtst.sh (on the
 Epoch_1 .log file) and analtst_conf.sh (on the Epoch_4 .log file).
 For example, for low conflict accuracy with the intact network, type
 "analtst.sh intact_e1.log." For the high conflict (e4) logs, use
 analtst_conf.sh instead of analtst.sh (simply because network error
 is indexed in a different column in the e4 logs and the analysis
 script has to know which column to average). This will output a list
 of four columns in a new file with the extension ".log.dt".  From
 left to right the columns are: epoch number, number of networks
 averaged across, mean number of errors (out of 2 possible for each
 test), and standard error.  For a general idea of how well a network
 has done, it is easiest to look at the last row in this list, after
 it has received all its training. If the network has an average error
 of roughly 1.0 (out of 2.0 max, this is 50%), it is performing at
 chance.  You should see that the intact network does not suffer
 nearly as much in accuracy from the high conflict (e4) condition,
 whereas the STN lesioned network performs well in low but not high
 conflict trials. DBS networks with external stimulation do not suffer
 as much as STN lesioned networks (for discussion, see supplement of
 Frank et al, 2007).


***PLOTTING LAYER ACTIVITY LEVELS DURING SELECTION***

 One last script that can be run on the .log files is called
 analcycles.sh. This script averages activity levels in the different
 layers of the network as a function of time (cycles), averaging unit
 activity across each layer indpendently, and across all
 networks. This enables you to plot the different layer activity
 levels as a function of time during response selection, using your
 favorite graphing program.  For example, type "analycycles.sh
 intact_c1.log."  This will output a file containing 16 columns, with
 the extension ".cyc.dt".  The first column of the output is cycle
 number, the second column is the number of trials that has been
 averaged across (across different networks and multiple trials for
 each network).  The third and fourth columns are mean and standard
 error of STN activity, respectively. Similarly, columns
 5,7,9,11,13,15 are mean GPe, GPi, Motor cortex, Thalamus, Output
 unit1, and Output unit2, respectively.  Each of these columns is
 followed by a column of the standard error for that layer.


*** SETTING TONIC DA LEVELS; OSCILLATIONS AND PARKINSON"S TREMOR ***


Tonic DA levels are set in the control panel. Default = .03 means that
this is the external input, which translates to around 0.5 activity
levels in the SNc units (but these are also noisy).  You should also
see that if you set tonic levels to 0, some oscillations should be
observed in STN and GP layers on some trials (corresponding to
Parkinson's tremor). You can look at these more carefully in the cycle
graph logs where activity levels of the various layers are
recorded. These oscillations are more reliably observed for high
conflict trials (due to greater overall input from cortex to STN,
which is what causes the rebound STN burst activity associated with
each oscillation (Frank, 2006)). The oscillations can also be more
reliably induced in a network with more competing alternatives, such
as the 4-alternative choice model in Frank (2006).