Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat

Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat Background: Operant hyperactivity, the emission of reinforced responses at an inordinately high rate, has been reported in children with ADHD and in the Spontaneously Hypertensive Rat (SHR), the most widely studied animal model of ADHD. The SHR emits behavior at hyperactive levels, relative to a normoactive strain, only when such behavior is seldom reinforced. Because of its dependence on rate of reinforcement, operant hyperactivity appears to be driven primarily by incentive motivation, not motoric capacity. This claim was evaluated in the present study using a novel strategy, based on the organization of behavior in bouts of reinforced responses separated by pauses. Method: Male SHR, Wistar-Kyoto (WKY) and Wistar rats (WIS) were exposed each to a multiple variable-interval schedule of sucrose reinforcement (12, 24, 48, 96, and 192 s) between post-natal days (PND) 48 and 93. Responding in each schedule was examined in two epochs, PND 58-62 and 89-93. Parameters of response- reinforcement functions (Herrnstein’s hyperbola) and bout-organized behavior were estimated in each epoch. Results: SHR emitted higher response rates than WKY and WIS, but only when rate of reinforcement was low (fewer than 2 reinforcers per minute), and particularly in the second epoch. Estimates of Herrnstein’s hyperbola parameters suggested the primacy of motivational over motoric factors driving the response-rate differential. Across epochs and schedules, a more detailed analysis of response bouts by SHR revealed that these were shorter than those by WKY, but more frequent than those by WKY and WIS. Differences in bout length subsided between epochs, but differences in bout-initiation rate were exacerbated. These results were interpreted in light of robust evidence linking changes in bout-organization parameters and experimental manipulations of motivation and response-reinforcement contingency. Conclusions: Operant hyperactivity in SHR was confirmed. Although incentive motivation appears to play an important role in operant hyperactivity and motoric capacity cannot be ruled out as a factor, response-bout patterns suggest that operant hyperactivity is primarily driven by steeper delay-of-reinforcement gradients. Convergence of this conclusion with theoretical accounts of ADHD and with free-operant performance in children with ADHD supports the use of SHR as an animal model of ADHD. Keywords: ADHD, Spontaneously Hypertensive rat, hyperactivity, operant, variable interval, bout Background school, poor interpersonal relationships, and psychologi- Attention deficit hyperactivity disorder (ADHD) is the cal problems such as depression and anxiety, among most commonly diagnosed childhood psychiatric disor- others [6-9]. The Spontaneously Hypertensive Rat (SHR) is the der, affecting between 2% and 12% of grade school chil- dren, and around 4% of adults [1-4]. It is characterized most widely used animal model of ADHD [10-12]. Evi- by difficulties related to impulsivity, inattention, and dence suggests that SHR displays the three main beha- hyperactivity [5]. ADHD is associated with problems in vioral characteristics of ADHD: impulsivity [13-15], inattention [16], and hyperactivity [17]. Nonetheless, the reliability of some of this evidence and the validational * Correspondence: Federico.Sanabria@asu.edu support it provides has been disputed [18,19]. Sanabria Department of Psychology, Arizona State University, P.O. Box 871104, Tempe, Arizona 85287-1104, USA © 2012 Hill et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 2 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 and Killeen [13] addressed the inconsistency of the evi- contribute to operant hyperactivity. The purpose of the dence regarding response inhibition deficits in SHR. present study is to advance the identification of such They concluded that, without a model of response inhi- processes by examining SHR performance at a high bition and appropriate procedures for estimating model level of detail. parameters, claims about impulsivity in SHR are unlikely Inferences on motor and motivational processes have to converge. The purpose of this paper is to extend this been drawn from performance in variable interval (VI) schedules of reinforcement. In these schedules, the first reasoning to another symptom of ADHD that is pre- response following an unsignaled interval of variable sumably expressed in the SHR: hyperactivity. duration is reinforced. Inferences are based on para- Hyperactivity in SHR has been assessed using the open-field method and the operant-conditioning meters of models fit to average response rates in VI method. The open-field method consists of measuring schedules [39,40]. Not all responses in VI schedules, the amount of locomotor activity (typically in the form however, are functionally equivalent, so averaging all of infrared beam breaks) in an enclosure [20]. Because responses in a session may neglect useful information activity measured by this method does not yield pro- [41]. In fact, the distribution of inter-response times grammed consequences, we refer to it as spontaneous (IRTs) provides additional information about the multi- activity. The operant-conditioning method consists of ple sources of variance in VI performance [42-47]. Fig- measuring the rate of emission of a target response ure 1 illustrates the IRT model used in the present (typically lever pressing), where the target response study, which we call the bout-and-pause model. Each occasionally produces a reinforcer (typically food). vertical line represents a response; the spaces between Because activity measured by this method operates on a vertical lines represent the IRTs. The critical assumption specific feature of the environment and yields a pro- of this model is that operant responding occurs in bouts grammed consequence, we refer to it as operant activity. separated by relatively long pauses [40,41,48]. Thus, Operant activity is often observed under interval sche- increased responding in the SHR at low rates of reinfor- dules of reinforcement, in which only the first target cement may be due to (1) faster responding within response following a programmed interval is reinforced. bouts, (2) longer bouts, or (3) shorter pauses between Interval schedules maintain an approximately constant bouts. The purpose of this study was to replicate past rate of reinforcement regardless of response rate [21], results that show that SHR hyperactivity is constrained thus isolating changes in rate of reinforcement from to low rates of reinforcement, and to characterize oper- ant hyperactivity in SHR in terms of bout-and-pause changes in activity. parameters. It is not clear that the SHR displays more spontaneous activity than control strains. Whereas some research has demonstrated spontaneous hyperactivity in the SHR [22-27], other research has shown this effect only at cer- tain ages [19,28], and still other research has not shown such an effect [29-31]. In contrast, operant hyperactivity is well demonstrated in the SHR [16,25,31-34]. Under interval schedules of reinforcement, the SHR typically responds at significantly higher rates than control strains [16,17]. An analogous difference has been observed between children with and without ADHD [35]. Performance under varying rates of reinforcement has been informative of the nature of operant hyperactivity in the SHR. Response rates in the SHR and control strains covary with rate of reinforcement, but the SHR responds at abnormally higher rates only when rate of Figure 1 Schematic timeline depicting the bout-and-pause reinforcement is low [32,33]. These researchers showed model of free-operant performance. A trial starts at the left-end of the timeline and progresses to the right. The time between trial that maximal responding was about equal for SHR and onset and the first response (vertical line labeled “Response 1”)is Wistar-Kyoto rats (WKY, which typically serves as con- the latency for that trial. The time between two consecutive trol strain), suggesting that superior motor ability alone responses is an inter-response time, or IRT. There are two types of cannot explain hyperactivity in the SHR. High rates of IRTs: within-bout (short) and between-bout (long). Thicker vertical reinforcement have also been shown to normalize the lines are bout-initiation responses; thinner lines are within-bout responses. The length of a bout is the number of responses operant performance of children with ADHD [36,37]. between latency and the first between-bout IRT, or between two These results suggest that non-motoric processes, such between-bout IRTs. as differences in responsiveness to incentives [38], may Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 3 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 lever presses when a force of about 0.2N was applied to Methods the lever. Responses were recorded with a 110-ms reso- Subjects lution. The house light was not turned on for experi- Eighteen male rats, 6 each of three different strains were mental sessions. A speaker located at the top of the side obtained from Charles River Laboratories (US) (Sponta- wall opposite to the food receptacle emitted 75 dB tones neously Hypertensive Rats, SHR; Wistar rats, WIS) and from a generator (ENV-223). Harlan Laboratories (US) (Wistar-Kyoto rats, WKY). The substrain of WKY bred by Harlan Laboratories has Procedure been shown to be the most appropriate normoactive control strain for the SHR [49]. Hopper training and autoshaping Experimental sessions were conducted daily at approxi- Rats arrived to the laboratory on post-natal day (PND) mately the same time of day for each rat. Hopper train- 24-25. Hopper training commenced on PND 39. Rats ing consisted of presenting a food pellet every 15 s on were pair-housed within their strains in a colony room average. After two days, rats were eating consistently with a 12:12 hour light:dark cycle; experiments were from the hopper and autoshaping of lever pressing conducted during the dark cycle. Rats were maintained started. Autoshaping consisted of extending a randomly at 85% of their free-feeding weights based on a logistic selected lever (left or right) for 10 s, every 30 s; a food function fitted to the growth curves provided by bree- pellet followed only right lever retractions. After 9 days ders. All rats were handled for a minimum of 2 min/day rats were consistently pressing the right lever each time by the researchers in the days preceding hopper train- it was extended. ing. Animals were weighed every morning, and fed a Operant Task supplementary amount of rodent chow every evening, at The first five minutes of each session served as an accli- least 12 hr before the following experimental session. At mation period, during which the houselight was off and the beginning of operant training the mean weights for the levers were retracted. After the acclimation period, the SHR, WKY, and WIS were 118, 127, and 190 g, the right lever was extended into the chamber, and a respectively. Home cages were always equipped with multiple variable interval (VI) schedule was in effect. water bottles. All handling procedures in the present One of five VI schedules (VI 12, 24, 48, 96, or 192 s) study were maintained according to the guidelines of was randomly selected. Schedules were implemented on the National Institute for Health, which were approved each trial by selecting without replacement from an 8- by the Arizona State University Institutional Animal item Fleschler-Hoffman distribution of intervals [50]; Care and Use Committee. the mean of the distribution was the nominal VI Apparatus requirement. Responses that occurred during the inter- val were recorded but had no programmed effect. Once All experimental sessions were conducted in 6 MED the selected interval time elapsed, the first lever press Associates modular test chambers (305 mm × 241 mm resulted in the delivery of one food pellet into the feed- × 210 mm). Each sound- and light-attenuating box con- ing aperture, which served as reinforcement. After each tained a ventilating fan. The fan provided a masking pellet delivery, the lever was retracted, a 5-s inter-trial noise of about 60 dB. The bottom of each box was lined interval (ITI) ensued, then the lever was extended again with a catch pan full of sanitary chip litter, and the floor and another interval was selected from the same VI dis- of each chamber had thin metal bars. The front and tribution. When an 8-item distribution was exhausted, back walls and the ceiling were made of clear polycarbo- the ITI was 20 s and another VI schedule was selected. nate; the front wall also served as a door. The food The five schedules of reinforcement were signaled by receptacle was attached to a square aluminum aperture one of five tones. Each tone (3-12 kHz) was presented (51 mm sides, 15 mm above the chamber floor), cen- on a unique on:off cycle (200-1000 ms) for the duration tered on the side wall against which the chamber door of the schedule. Sessions ended when every schedule was latched. Activation of the food dispenser released was implemented once, or after 70 minutes, whichever one 45-mg food pellet (Dustless Precision Pellets , happened first. Fifty-four daily sessions were conducted, Rodent Grain-Based Diet, Bio-Serv, Frenchtown, NJ). 7 days/week. Although the study only involved use of the right lever Measures (closest to the door of the chamber), two retractable The first analysis was based on two measures: response levers (MED associates, ENV-112CM) were on either rate and reinforcement rate. Response rate was com- side of the food hopper. The inside edge of each lever puted for each VI schedule as the number of responses was 8 mm from the closest vertical edge of the recepta- emitted while the schedule was effective, divided by the cle. A Med-PC interface connected to a PC computer ran Med-PC IV software. This computer recorded time the schedule was in effect (excluding ITIs). Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 4 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Reinforcement rate was computed for each VI as the number of reinforcers collected at each schedule, divided by the time the schedule was in effect (excluding ITIs). Response rates were measured in two epochs, PND 58-62 (epoch 1) and PND 89-93 (epoch 2), follow- ing approximately 10 and 40 sessions of VI training, respectively. Epoch 1 corresponds to a conservative esti- mate of early adulthood, but possibly captures late ado- lescence. Epoch 2 corresponds to adulthood [51]. Herrnstein’s (1970) hyperbola parameters were esti- mated on the basis of response rates (explained in Results section) [40]. In a subsequent analysis, response rates were further analyzed by separating response latencies from inter- response times (IRTs). The distinction between latencies and IRTs is depicted in Figure 1. Latencies were the intervals between trial onset (lever extension) and the first lever press in that trial. IRTs were the intervals between consecutive lever presses within the same trial. Latencies were classified in two groups: the first latency in each VI (Latency 1), and all subsequent latencies within the same VI (Latencies 2-8). This classification took into account that, within each VI schedule, the duration of the first interval to reinforcement could only be cued by the discriminative tone, whereas the duration of subsequent intervals could also be cued by the dura- tion of the preceding intervals. Median Latencies 1 and 2-8 were computed separately for each rat within each Figure 2 Mean (± SEM) response rates of each strain (SHR: unfilled squares; WKY: filled circles; WIS: filled triangles) as a VI schedule and epoch, and then averaged within strain function of mean rate of reinforcement, in two epochs: PND (mean median latencies). Estimates of bout-initiation 58-62 (epoch 1; top panel) and PND 89-93 (epoch 2; bottom rates, within-bout response rates, and bout length-the panel). Response rate increased with rate of reinforcement in all parameters of the bout-and-pause model-were based on strains and epochs. SHR response rates were higher than those of the distribution of IRTs (explained in Results section). WKY and WIS when reinforcement was delivered less than twice per minute. At higher rates of reinforcement, WIS response rates were lower than those of SHR and WKY. Curves through the data are Results traces of Herrnstein’s hyperbola (Equation 1). Figure 2 shows mean (± SEM) response rates of each strain as a function of rate of reinforcement in each epoch. Response rates of SHR are indicated by unfilled more noticeable at higher rates of reinforcement, and squares, WKY by filled circles, and WIS by filled trian- forWIS therewas virtually no changein responserate gles. Visual inspection of Figure 2 reveals a positive cor- with age. To characterize these patterns of response relation between response rate and rate of rate, we estimated the parameters of Herrnstein’s (1970) reinforcement in all strains. Differences in response hyperbola and compared them across strains [40]. rates between strains and across rates of reinforcement are visible in both epochs. When rates of reinforcement Herrnstein’s hyperbola were low (fewer than 2 responses per minute), SHR Herrnstein (1970) extended the Matching Law [52] to responded at a higher rate than other strains. At higher describe the relation between response rate (B)and rate rates of reinforcement, SHR and WKY response rates of reinforcement (R) on a single operandum. Herrn- converged, and WIS response rates remained low (40-50 stein’s rationale was that all the responses other than responses per minute). These patterns of response rate the target response are reinforced at an unknown rate. across strains and schedules were visible in epoch 1 and Such rate, however, may be estimated if it is assumed were magnified in epoch 2. SHR response rate increased that (a) the ratio of two response rates matches the ratio with age regardless of rate of reinforcement, whereas for of the corresponding reinforcement rates (Matching WKY age-dependent increases in response rate were Law), and (b) the target response rate and the non- Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 5 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 target response rate add to a constant k. Under such where n is the number of observations (n =5sche- assumptions, dules × 18 rats = 90 observations in each epoch), RSS is the minimized residual sum of squares obtained from kR fitting a hypothesis to the data, and c is the number of B = , (1) R + R free parameters in the hypothesis. c can also be com- puted as the degrees of freedom of the estimates of the where R is the estimated rate of reinforcement pro- overall means of k and R plus 1 parameter for error vided by non-target responses. When reinforcement is variance, i.e., 2 parameters × 18 rats - the number of programmed on VI schedules, R typically falls only constraints + 1. In the preceding example, c =36-3+ slightly below programmed reinforcement rates and 1 = 34 free parameters. thus serves as the independent measure; B is the depen- Note that AICc increases with RSS and with c;there- dent measure; k and R are free parameters. Equation 1 fore smaller AICc are indicative of close fit to the data predicts that responding increases at a negative pace as and parsimony. Hypotheses with smaller AICc were reinforcement increases, with asymptote k. R is the rate favored over those with higher AICc. ΔAICc was com- of reinforcement at which response rate reaches half of puted as the difference between each the AICc of its asymptote (i.e., when R = R ,B = k /2). hypothesis i and the lowest AICc among all hypotheses Following Herrnstein’s (1970) rationale [40], k is often (ΔAICc =AICc -AICc ). The hypothesis with few- i i MIN interpreted as a maximum limit on motoric perfor- est free parameters among those with ΔAICc < 4 was mance, influenced only by response characteristics; R is selected as the best description of the data. This selec- interpreted as indexing motivation for the reinforcer, tion was conducted separately for the 2 epochs in which influenced only by reinforcer characteristics [39]. A data were collected. large body of evidence supports Equation 1 as an accu- Table 1 shows the 5 hypotheses with the lowest rate characterization of response-reinforcement func- ΔAICc in each epoch. The selected hypothesis for epoch tions like those in Figure 2[53-56]. The empirical 1 assumes that R = R , and all other parameters eWIS eSHR support for motoric/motivational interpretations is, varied between strains. For epoch 2, the selected however, somewhat mixed [57]. hypothesis assumes different parameters for each strain. Parameters of Equation 1 were estimated by fitting Figure 3 shows the mean estimates of k and R for each Equation 1 to the data of each individual animal, in epoch based on the selected hypotheses. It was inferred each epoch, using the method of least squares. Para- that k >k >k in both epochs, which indicates WKY SHR WIS meters k and R were assumed constant across values of that WKY had the highest asymptotic response rates, R, but could vary between rats, thus yielding 2 × 18 = followed by the SHR and then WIS. Mean k estimates 36 model parameters. Comparisons were conducted increased across epochs for all strains. In epoch 1, between mean estimates of each strain, henceforth R >R = R ;atepoch 2, R >R >R . eWKY eWIS eSHR eWKY eWIS eSHR referred to by the parameter and strain abbreviation: k , k , k , R , R ,and R . The curves SHR WKY WIS eSHR eWKY eWIS Table 1 Hypotheses of VI performance with lowest ΔAICc. in Figure 2 are traces of Equation 1 using the mean esti- Hypothesis c RSS ΔAICc mates of k and Re for each strain. Various constraints were imposed on model para- Epoch 1 (PND 58-62) meters to draw inferences on between-strain differences. R = R 36 306.78 0.00 eSHR eWIS These constraints consisted of holding constant the None 37 306.12 5.49 mean estimate of either model parameter across all, k =k 36 458.24 36.11 WIS WKY some, or none of the strains. Each particular combina- k = k , R = R , 35 496.05 37.78 SHR WIS eWIS eWKY tion of constraints constituted a hypothesis. Thus, for k = k 36 495.73 43.19 SHR WIS example, k ≠ k = k , R = R = R is Epoch 2 (PND 89-93) SHR WKY WIS eSHR eWKY eWIS the hypothesis that mean k varied between SHR and None 37 581.35 0.00 WKY, but not between WKY and WIS, and mean R R = R 36 738.84 15.89 eWIS eWKY did not vary between strains. There were 15 possible k = k , R =R , 35 837.20 21.67 SHR WIS eWIS eWKY constraint combinations. k = k 36 807.35 23.87 SHR WIS Hypothesis testing was conducted separately in each R = R 36 904.42 34.09 eSHR eWIS epoch, using the corrected Akaike Information Criteria Note. The label of each hypothesis stipulates the constraints on mean parameter estimates. c is the number of free parameters in each hypothesis; (AICc) [58], RSS is the residual sum of squares from model fitting. For all hypotheses, the number of observations was n = 90. See Equation 2 and text for computation 2nc of ΔAICc. Hypotheses are arranged according to ΔAICc within each epoch. (2) AICc = n ln RSS/n + n − c − 1 Hypotheses with ΔAIC = 0 were selected for parameter estimation. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 6 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 select an unintelligible model [58]. Conventional approaches, such as null-hypothesis testing, are not designed for this task: falsifying the null hypothesis that a particular component did not vary between strains in one or more schedules provides little information on the contribution of that component to differences in response rate. Therefore, the analysis presented here is qualitative; inferences drawn from this analysis should be taken as exploratory and provisional, pending empiri- cal verification. Latencies Figure 4 shows mean (± SEM) median Latency 1 and Latencies 2-8 for each strain in each epoch, as a func- tion of rate of reinforcement. Latency 1 (left panels) did not vary systematically with rate of reinforcement in Figure 3 Mean (± SEM) estimates of Herrnstein’shyperbola either epoch or across strains in epoch 1. In epoch 2, parameters k (asymptotic target response rate; left panels) and R (rate of reinforcement of non-target behavior; right panels) mean median Latency 1 was longer for WKY than for for each strain in epochs 1 and 2 (top and bottom panels, the other strains, regardless of rate of reinforcement. respectively). Estimates are based on hypotheses selected The right panels of Figure 4 and their insets show that according to AICc (Table 1). Estimates of k for SHR were Latencies 2-8 declined with rate of reinforcement. In intermediate relative to other strains. Estimates of R for SHR were epoch 1, Latencies 2-8 were mostly undistinguishable low relative to other strains, and approximately constant across epochs. between strains, with the possible exception of the longer latencies of WKY at the lowest rate of reinforce- ment. In epoch 2, median Latencies 2-8 of WIS were R does not appear to change across epochs, whereas eSHR longer on average, but also more variable across rats, R and R increased. eWKY eWIS than those of SHR and WKY. Also in this epoch, when Inferences from Herrnstein’s hyperbola parameters rate of reinforcement was less than 1 per minute, mean suggest that instrumental overactivity could be attribu- median Latencies 2-8 were about 1 s shorter for SHR ted to a higher motivation for the reinforcer, which did than WKY. The slopes of rescaled Latencies 2-8 (each not decline over nearly 30 days that separated the two median latency was divided by the median latency in VI assessment epochs, and not to differences in motoric 12 s, then logged, base 2), shown in the insets, reveal a capacity. This analysis, however, was based on average within-subject sensitivity of Latencies 2-8 to rate of rein- response rates in each VI schedule, which conflate two forcement in both epochs. This sensitivity was more types of intervals within the denominator: response pronounced in WKY than in the other strains. latencies and inter-response times (IRTs). Because rodent VI performance is typically organized in bouts Inter-response times (IRTs): Bout-and-pause model [42], IRTs may be further disaggregated into between- To account for the distribution of IRTs in each sche- bout and within-bout IRTs. Latencies, between- and dule, response rate in each VI schedule, excluding laten- within-bout IRTs may each depend on a distinct set of cies, was disaggregated into bout-initiation rate (the variables [45], which may further inform the sources of reciprocal of the mean IRT separating response bouts) SHR overactivity. and within-bout response rate (the reciprocal of the In the next two sections we examine the components mean IRT within bouts). This disaggregation consisted of response rate in SHR, WKY, and WIS. This analysis of estimating the parameters of a bi-exponential density is aimed at identifying candidate components that may function by fitting it to the distribution of IRTs in each account for the differences in response rate between VI schedule. The density function is SHR and WKY selectively at low rates of reinforcement, −w(t−0.11) −b(t−0.11) and between SHR and WIS at all rates of reinforcement. p(IRT = t)= pwe +(1 − p)be , b ≤ w ≤ 9s;0 ≤ p ≤ 1 (3) An AIC-basedanalysisappearsto bebestsuitedto where p is the proportion of IRTs within bouts; 1/(1 - address this goal, because the relation between response p) is the mean bout length, measured in lever presses. w rate and its components is not linear (see Appendix). is theresponseratewithin bouts; b is the rate at which Without an apriori selection of hypotheses, however, bouts are initiated. Because responses take a minimum the combinatorial of parameters and factors implies a time to be produced, that minimum time (the shortest computationally intractable analysis that may ultimately Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 7 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Figure 4 Mean (± SEM) median Latency 1 (first latency within each VI schedule; left panels) and Latency 2-8 (right panels) as a function of mean rate of reinforcement, for each strain in epochs 1 and 2 (top and bottom panels, respectively). The insets of the right panels are mean rescaled latencies: each individual median Latency 2-8 in each schedule was divided by the median Latency 2-8 in VI 12 s of the same rat, and then logged (base 2). Thus, a rescaled latency of 3 indicates that the median latency in that VI (excluding Latency 1) was 2 = 8 times longer than in VI 12 s. For all strains, Latency 1 did not vary systematically with rate of reinforcement, whereas Latencies 2-8 were shorter with higher rates of reinforcement. SHR latencies were generally undistinguishable from those of the other strains, with the possible exception of the shorter SHR Latencies 2-8 when reinforcement was delivered less than once per minute. possible IRT) must be subtracted from the duration t of previously to illustrate differences in pause and bout every IRT [45]. The minimum IRT recorded for every responding in both rats and pigeons [42,60]. Often, rat was 0.11 s, which was the resolution at which these plots take on a “broken-stick” appearance with a responses were recorded. Therefore, 0.11 s were sub- steeply declining initial left limb and a more gradually tracted from t in the exponents of Equation 3, and w declining right limb. A long initial limb on the leftmost was constrained to be less than or equal to 1/0.11 ≈ 9 side of the graphs indicates a high proportion of within- responses per second. bout responses, p. The slope of the left limb is the Parameters of Equation 3 were estimated for each within-bout rate of responding, w; the slope of the right individual rat in each epoch using the method of maxi- limb is the rate of bout initiation, b. The curves in Fig- mum likelihood [59]. Figure 5 shows semi-log survival ure 5 show that Equation 3 provided a good fit of the plots of IRTs in each schedule and epoch, averaged data, although the broken-stick pattern was most clearly within each strain. These plots have been used visible in WKY at low rates of reinforcement. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 8 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Figure 5 Semi-log survival plots showing the mean proportion of IRTs greater than t in each schedule (symbols), strain (columns), and epoch (rows). Proportions were calculated for each rat in bins that contain, each, 1% of the IRTs; binned proportions were then averaged over rats. Curves through the data are the mean traces of the bout-and-pause model (Equation 3), drawn using maximally likely individual estimates. SHR survival functions (left panels) were steeper and more linear than those of WKY (center panels), indicating, respectively, shorter IRTs (higher response rate) and responses organized in less distinct bouts. Mean (± SEM) bout-and-pause parameter estimates rate of reinforcement. At the lowest rate of reinforce- for each strain at each VI schedule and epoch are ment, w =0.80and w = 1.17 responses per sec- WKY WIS shown in Figure 6. To compute the mean estimates of ond; at the highest rate of reinforcement, w =1.04 WKY w and b, individual estimates were weighed by p and (1 and w = 1.31 responses per second. These trends WIS - p), respectively, because confidence on w and b esti- were dwarfed by the large between-subject and mates co-varies with these weights. The top panels of between-schedule variability in estimates of w . More- SHR Figure 6 show the mean estimates of p, w and b in over, in every schedule, w >w >w (mean SHR WIS WKY epoch 1; the bottom panels show estimates in epoch 2. across schedules = 2.09, 1.24, and 0.96 responses per Mean bout-and-pause parameter estimates are labeled second, respectively). It is important to note, however, in the same way as Herrnstein’s hyperbola estimates (e. that estimates of w and w were based on 2-3 rats SHR WIS g., p , p , p ). of each strain, because p =0formost of theserats in WKY SHR WIS Estimates of the proportion of within-bout IRTs (p)in most schedules. epoch 1 are shown in the top-left panel of Figure 6. Estimates of bout initiation response rate (b)inthe Estimates of p were substantially higher (mean first epoch are shown in the top-right panel of Figure 6. WKY across schedules = .90) than those of p (.17) and Estimates of b and b systematically increased with SHR SHR WIS p (.26). This means that WKY produced substantially rate of reinforcement. At the lowest rate of reinforce- WIS longer bouts than SHR and WIS. Moreover, whereas ment, b = 0.36 and b = 0.32 responses per second; SHR WIS p and p were relatively constant across rates of at the highest rate of reinforcement, b =0.85and SHR WIS SHR reinforcement, p increased with higher rates of rein- b = 0.63 responses per second. Estimates of b var- WKY WIS WKY forcement, from .80 at the lowest rate to .99 at the high- ied as an inverted-U function of rate of reinforcement, est rate. peaking at the second highest rate of reinforcement Estimates of within-bout response rate (w) in epoch 1 (0.23 responses per second). In every schedule, b SHR areshown in thetop-middlepanel of Figure6. Esti- >b >b (mean across schedules = 0.61, 0.49, and WIS WKY mates of w and w increased only slightly with 0.15 responses per second, respectively). WKY WIS Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 9 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Figure 6 Mean (± SEM) estimates of bout-and-pause parameters (Equation 3) as a function of mean rate of reinforcement, for each strain (SHR: squares; WKY: circles; WIS: triangles) in epochs 1 and 2 (top and bottom panels, respectively). In all epochs and schedules, SHR emitted shorter but more frequent bouts (low p in left panel, high b in right panel) than WKY. During epoch 1 SHR emitted faster within- bout responses (high w in top-center panel), but estimates of w were only possible for 2-3 out of 6 SHR rats; estimates of w were possible for all 6 WKY. The bottom panels of the Figure 6 show the mean (± Estimates of b in the second epoch are shown in the SEM) estimates of p, w and b in epoch 2. The bottom- bottom-right panel of Figure 6. As in the preceding left panel of Figure 6 shows that, similar to those in the epoch, b >b ≥ b in every VI schedule, and SHR WIS WKY preceding epoch, estimates of p in epoch 2 were estimates of b also increased with rates of reinforce- WKY very high and increased further with rate of reinforce- ment, including b . Estimates of b and b WKY SHR WKY ment. Estimates of p and p increased between increased between epochs in every VI schedule; b SHR WIS WIS epochs in every VI schedule. The increase was particu- remained relatively unchanged. larly noticeable in p ; on the average, estimates of SHR p more than tripled between epochs. Like in the pre- Discussion SHR ceding epoch, however, there were no trends in p Operant hyperactivity was observed in SHR, particularly SHR and p across VI schedules comparable to those of during adulthood (epoch 2, PND 89-93), but only at low WIS p . rates of reinforcement (less than 2 and 4 reinforcers per WKY Estimates of w in epoch 2 are shown in the bottom- minute on PND 58-62 and 89-93, respectively). Esti- middle panel of Figure 6. Estimates of w and w mates of Herrnstein’s hyperbola parameters (k, R ;see WKY WIS e increased between epochs in every schedule, but only Equation 1) suggest that operant hyperactivity in SHR is w preserved its positive correlation with rate of rein- not due to enhanced motor capacity relative to WKY WKY forcement. Estimates of w remained relatively high, (the converse is most likely the case: k >k ). SHR WKY SHR particularly when rate of reinforcement was low. Instead, highly valued activities-such as searching for Between-subject variance in individual estimates of w food-are less likely to be displaced by less valued, com- SHR increased substantially between epochs, further dwarfing peting activities in SHR than in WKY (R <R ). eSHR eWKY any differences between strains. The increase in This finding is consistent with the notion that frequent between-subject variance was due to 2 SHR with unde- reinforcement normalizes free-operant ADHD perfor- termined w in epoch 1, whose individual w estimates, mance [36,37]. averaged over VI schedules in epoch 2, were 5.50 and Differences between WKY and SHR in response rate 8.89 responses per second. Such high estimates were and in estimates of Herrnstein’s hyperbola parameters not obtained for any other rat of any strain. replicate prior findings in adult rats [32,33], and Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 10 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 generalize them, to a limited extent, to younger rats. the other 2 strains, particularly in epoch 1. SHR bouts Such generalization suggests that the inferences drawn became longer in adulthood, but were still systematically from the more detailed analysis, based on bout-and- shorter than those of WKY, regardless of schedule. The pause parameter estimates, are not idiosyncratic to the length and density of WKY bouts, unlike those of other present data, but reflect a more general phenomenon. strains, increased with increasing rate of reinforcement, Unlike Herrnstein’s hyperbola, the bout-and-pause which may explain why differences in overall response model does not assume the functional equivalence of all rate between SHR and WKY are confined to low rates operant responses. Instead, the bout-and-pause model of reinforcement. In epoch 1, WIS parameters were gen- erally intermediate to those of SHR and WKY across supports an analysis based on latency and IRT statistics that separates responses into more meaningful func- parameters; in epoch 2, WIS maintained relatively short tional categories, which we examine next. bouts that contained few responses. The bout-and-pause analysis thus identifies the higher Latencies frequency of bouts in SHR as the main source of hyper- Latencies shown in Figure 4 suggest that tones did not activity, and higher within-bout rate as a possible sec- support the discrimination between schedules of reinfor- ondary source. Rate of bout initiation is particularly cement. Instead, it appears that latencies were updated sensitive to motivational manipulations, increasing as a according to preceding intervals to reinforcement. Such function of reinforcement deprivation and availability in a process is evinced in the steeper Latency 2-8 curves rats [42-45], mice [61], and pigeons [62]. This correla- (right panels) relative to Latency 1 curves (left panel). tion suggests that SHR hyperactivity is caused by a When the interval to reinforcement was not cued by hypermotivation to incentives. Such inference is consis- prior intervals (Latency 1), WKY took longer to emit tent with latency patterns in adulthood, to the extent the first response, relative to SHR and WIS, but only in that latencies are indicative of motivation [63,64], and adulthood. In subsequent intervals (Latencies 2-8), WKY with inferences drawn from Herrnstein’s hyperbola para- latencies became particularly sensitive to rate of reinfor- meters, both here and in prior studies [32,33]. cement, especially in adulthood. Note that, in adulthood, the pattern of Latencies 2-8 (bottom-right panel) is a A delay-of-reinforcement-gradient hypothesis vertically-flipped analogue of response rates (Figure 2, Brackney and colleagues [45], however, caution against a bottom panel). This indicates that adult Latencies 2-8 straightforward interpretation of changes in rate of bout became either (1) an important determinant of response initiation in terms of incentive motivation. Based on the rate,or(2) sensitivetowhatever factors determined performance of Sprague-Dawley rats, they concluded response rate. Alternative (1) does not appear to be the that changes in bout initiation rate alone may be inter- case: based on mean adult performance of each strain, preted as changes in incentive motivation, but when for every latency there were about 11 - 20 IRTs in VI 12 such changes are accompanied by changes in other s, and up to 58 - 150 IRTs in VI 192 s. That is, latencies parameters, they may reflect non-motivational processes contributed between 0.67% and 8% of the response rate that only indirectly impact motivation. For instance, a denominator. Although latencies appear to reflect pat- tandem ratio requirement at the end of the VI lengthens terns of hyperactivity in SHR, particularly in adulthood, bouts and increases the number of responses within they cannot be the main source of these patterns. That them, but also reduces the frequency of bouts [42-45]. source is, therefore, most likely to be identified in the The latter effect cannot be accounted for by a reduction distribution of IRTs. in incentive motivation, because the tandem require- ment does not change the rate of reinforcement sub- IRTs stantially. Instead, Brackney and colleagues suggested The bout-and-pause model assumes that responses are that the tandem requirement favors the reinforcement organized in two separate categories: responses that of long response bouts [65], concomitantly reducing the initiate bouts and responses emitted within bouts. An temporal contiguity between bout initiations and rein- analysis based on bout-and-pause premises suggests that forcement. It is hypothesized that reduced initiation- SHR hyperactivity reflects a high rate of bout initiations reinforcement contiguity results in less effective reinfor- (higher b) in SHR relative to WKY (Figure 6, right cement of bout initiation and a consequent reduction in panels), even though SHR response bouts were shorter its rate. (lower p; Figure 6, left panels). Because of the short Brackney and colleagues’ [45] account of how a tan- length of their bouts, estimates of SHR within-bout dem requirement reduces bout initiation rate implies response rate (w) were not reliable. Nonetheless, the that the effectiveness of reinforcement declines with the performance of those SHR with p > 0 suggests that SHR temporal distance between the reinforced response (bout initiation) and the reinforcing event (food) [66,67]. within-bout response rates were higher than those of Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 11 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 The slope of this decline in reinforcement effectiveness children with ADHD [77]. The critical evidence was col- is the delay-of-reinforcement gradient [68,69]. The effect lected using concurrent independent VI schedules of of flattening the delay-of-reinforcement gradient on reinforcement, where two schedules, like those used in operant performance should be similar to the effect of the present study, were simultaneously in effect In this imposing a tandem requirement: It should facilitate the context, Shull [47] has argued that the length of a “visit” reinforcement of long bouts while reducing bout fre- to either schedule is functionally equivalent to the quency. Compared to the initiation of short bouts sup- length of bouts in a single-schedule design (following Herrnstein’s rationale, single-schedule designs may be ported by steeper gradients, the initiation of long bouts thought of as concurrent-schedule designs where one supported by flatter gradients should be less frequent because flatter gradients envelop more competing schedule is implicit [40]). Taylor, Lincoln and Foster responses (between-bout activities, within-bout [77] reported that children with ADHD switch more responses) than steeper gradients. These intuitions are between concurrent VI schedules, and thus produce diagrammed in Figure 7. shorter visits, than non-ADHD controls, as long as Compared to SHR, WKY displayed long-but-infre- switching between schedules is not penalized with a quent response bouts (high p and low b in Figure 6) in changeover delay. The converging patterns of free-oper- both assessment epochs. This pattern suggests that one ant performance in SHR and in children with ADHD important source of SHR hyperactivity is the steepness suggest that (1) the emission of short free-operant bouts of its delay-of-reinforcement gradient. This hypothesis is may be a diagnostic feature of the behavioral phenotype consistent with prior SHR data [34,38,70-73] and with of ADHD, revealing a deeper deficit in learning observations of children with ADHD [74]. The charac- response-reinforcement contingencies, and that (2) the terization of ADHD in terms of steeper delay-of-reinfor- SHR models these attributes of ADHD, further confirm- cement gradients is a core assumption of the dynamic ing its utility as an animal model of ADHD [13]. developmental theory of ADHD [38,75,76]. Regarding WIS rats, the intermediate length (in epoch 1) and fre- Alternative sources of hyperactivity quency (in both epochs) of their bouts suggest an inter- SHR produced shorter bouts at a higher rate than WKY mediate delay-of-reinforcement gradient for this strain over the range of VI schedules tested in the present relative to SHR and WKY. study. Prior research [42-45] suggests an interpretation The emission of short free-operant bouts, which sup- of these differences in terms of delay-of-reinforcement ports the delay-of-reinforcement-gradient hypothesis of gradients. Based on such interpretation, it would be operant hyperactivity, has recently been observed in expected that appending a tandem ratio requirement to the VI schedule of SHR would reduce the difference in its performance relative to WKY. But aside from length- ening bouts and reducing their frequency, a tandem requirement also increases within-bout responding, which has the net effect of increasing overall response rate [42,44,45]. That is, the tandem-ratio “treatment” is expected to increase SHR activity, not decrease it. This means that, although steeper delay-of-reinforcement gradients may be the main source of SHR overactivity, it is unlikely to be the only one. Two additional sources are possible: Increased motor capacity in SHR Estimates of Herrnstein’s hyperbola parameters ruled Figure 7 Delay-of-reinforcement-gradient hypothesis of SHR out motor capacity as a source of SHR hyperactivity, on hyperactivity. Ticks on the x-axis are responses; thick ticks are bout the basis of projected asymptotic response rates (k). It initiations. A reinforcer is delivered after the last response on the right of each panel. The sloped curves indicate that reinforcement is appears intuitive that such asymptotic rates reflect more effective with temporal proximity to the reinforcer. For SHR, motoric constraints in performance. Nonetheless, SHR only short bouts are effectively reinforced (panel A); for WKY, longer andWIS within-boutresponses (the faster response bouts are reinforced (panel B). Note that reinforcement affects more class) constituted only about half of the responses at the behaviors in panel B than in panel A, which entails that a smaller highest rate of reinforcement, when response rates were proportion of reinforcement strengthens bout initiation in panel B than in panel A. Therefore, relative to competing behaviors such as nearly asymptotic. This means that SHR and WIS could activities between bouts and responses within bouts, bout respond faster than what Herrnstein’s k suggests. Para- initiations are less effectively reinforced when reinforcement meter w is probably a more realistic reflection of moto- gradients are flatter. ric constraint. Estimates of the highest within-bout Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 12 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 response rate across schedules (Figure 6, middle panels) multiple-schedule context. Such close resemblance sug- suggest somewhat higher motoric capacity in young gests that differences between SHR and WKY perfor- SHR (PND 58-62), and limited motoric capacity in adult mance are robust against confounding factors in the WIS (PND 89-93). Although estimates of w are compro- present study. Second, the critical differences in bout- mised by the short bouts produced by SHR and WIS, and-pause parameters between strains (high p in WKY, elevated motoric capacity may not be ruled out as a high b is SHR) were not schedule-dependent. potential source of SHR hyperactivity, at least not dur- Differentiation between within-bout response rate and ing the transition from adolescence to adulthood. bout-initiation rate Furthermore, the notion that motoric capacity is Of the 90 rat × schedule estimations of bout-and-pause involved in SHR hyperactivity is consistent with prior parameters in each epoch, 46 in epoch 1 and 20 in data showing that IRTs shorter than 0.4 s are more fre- epoch 2 yielded p = 0 or 1. In those cases, the distribu- quently emitted by adult SHR than by adult WKY in VI tion of IRTs did not resemble a mixture of two expo- 30 s [34]. nentials (Equation 3) but just a single exponential. This SHR hypermotivation is noticeable in the nearly linear (in logarithmic scale) Although the interpretation of differences in bout-initia- IRT survivor plots shown in Figure 5, particularly those tion rate (b) in terms of incentive motivation is condi- of SHR and WIS and of rich schedules. Exponential IRT tional to the absence of changes in other parameters distributions yielded ambiguous estimates of p and inde- (see A delay-of-reinforcement-gradient hypothesis), terminate estimates of either w or b (footnote 1 clarifies changes in p and w do not rule out differences in incen- how it was chosen between p = 0 and p = 1 in each esti- tive motivation. In fact, it appears that the reduction in mation). Despite consistent differences in parameters b that is expected from a tandem ratio treatment would across schedules and strains, the uncertainty regarding be too small to reduce SHR estimates to WKY levels: parameter estimates implies that inferences drawn from Brackney and colleagues [45] report that a tandem ratio them, particularly in epoch 1, should be taken with cau- requirement reduced b by 37% in a VI 120 s. In a com- tion. Although the short length of SHR bouts is itself a parable schedule (VI 96 s), estimates of b in WKY were very important finding, future research should promote 71% shorter than SHR in epoch 1, and 78% shorter in longer bouts by imposing small tandem ratio require- epoch 2. Therefore, it seems likely that incentive moti- ments to all strains. This methodological adjustment vation differences in b by itself-also contributed to oper- would make pauses between and within bouts more ant hyperactivity in SHR. readily distinguishable. Confound of training experience and age Limitations The parameters of Herrnstein’s hyperbola and the bout- Finally, we acknowledge and address three potential lim- and-pause model were examined in two epochs, PND itations of the present study. These limitations do not 58-62 and 89-93. Between epochs, response rates compromise the basic conclusions inferred from the increased across schedules in SHR, only at high reinfor- data, but constrain the interpretation of the present cement rates in WKY, and not visibly in any schedule in results in terms of underlying psychological and devel- WIS. These divergent patterns of change over time exa- opmental processes. cerbated the differences in response rate between SHR Lack of stimulus control and control strains at low rates of reinforcement. The The first latency in each VI component (Latency 1 in elevated rate of weakly reinforced responses is the signa- Figure 4) did not vary systematically with rate of reinfor- ture of operant hyperactivity in SHR [32,33]. One possi- cement for any strain. This indicates that the tone asso- ble implication of the present results is that ciated with each VI schedule was not effective in hyperactivity emerges more strongly with adulthood. controlling response rate. Differences in response rate Although past research is consistent with these results across schedules depended on adjustments of response [34], they do not appear to be consistent with the modal rate to local rate of reinforcement. Such adjustments developmental trajectory of hyperactivity in ADHD may have introduced extraneous variability in VI perfor- [78-80]. Impulsive-hyperactive symptoms associated mance among strains. Furthermore, even if the tones with ADHD generally decline with age. Note, however, had been effective discriminative stimuli, schedule inter- that thepresent studydid notexamine ageseparately actions might have also confounded our results. These from training experience: older, more hyperactive SHR limitations, however, do not appear to seriously compro- had more exposure to the schedules of reinforcement mise the findings of the present study, for two reasons. than younger, less hyperactive SHR. The inconsistency First, the changes in overall response rate as a function between hyperactivity in SHR and in ADHD may stem of rate of reinforcement and strain resemble those from this confound. Our data, in fact, points at a possi- ble coincidence between the developmental trajectories observed before [32,33], which were not collected in a Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 13 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 of SHR and ADHD hyperactivity: SHR learned to (or solving for B, matured to, we do not know) produce longer response 1 1 L bouts, a pattern that was more typical of WKY controls. (A3) B = + − . T t Tt Although this study was not primarily aimed at discri- minating between practice and maturational effects on Based on the assumed distribution of IRTs (Equation operant hyperactivity, it provides, nonetheless, hints that 3), and assuming a minimum response duration δ (0.11 may guide future research on developmental factors s in this report), the mean IRT is involved in ADHD. p (1 − p) t = + + δ. (A4) Conclusions w b This study confirms that operant hyperactivity in SHR, Mean response rate in a VI schedule may thus be a purported animal model of ADHD, is expressed only recovered by substituting t in Equation A3 with the at low rates of reinforcement. This effect was observed right-hand side of Equation A4, and assuming that T in the transition from adolescence to adulthood (PND equals the VI requirement I. A more precise estimate of 58-62) and, more markedly, during early adulthood T is (PND 89-93). A close examination of the microstruc- ture of VI performance indicates that, across ages and T = I + . (A5) schedules, operant hyperactivity in SHR may be due to steeper delay-of-reinforcement gradients relative to control strain WKY. Inordinate motivation for incen- Endnotes tives and elevated motoric capacity may also contribute Theestimateof p for several rats under various VI to operant hyperactivity in SHR. With adulthood, schedules was 1.0 or zero, which posed a problem for delay-of-reinforcement gradients in SHR appear to flat- parameter estimation. Whether p =1.0 or zero,or w = ten; its motoric capacity becomes hardly distinguish- b, Equation 1 is reduced to an exponential density func- able from WKY, but its motivation for highly valued tion, with either p =1.0,and b not computable (i.e., incentives, such as sucrose pellets, grows even stron- bouts are infinitely long) or p =0,and w not computa- ger. Whether these changes in performance parameters ble (i.e., bouts are 1 lever press long). These two situa- are due to training experience, maturation, or a combi- tions are not distinguishable. When p =1.0 or p =0 nation of both, is yet unclear. These results suggest, had to be chosen for a particular rat, the variance with nonetheless, that complex and important learning, respect to p estimates in other VI schedules within the motivational, and developmental processes expressed same subject were taken into consideration. The esti- in SHR behavior appear to underlie operant hyperac- mate of the ambiguous p was the one that minimized tivity in ADHD. thevarianceamong p estimates. When p could take either value, 1.0 or zero, in all 5 VI schedules (this hap- Appendix: Computing mean response rate from pened in 7 of 36 rat × epoch observations), p was invari- its components ably estimated to be zero, because under such Response rate B over an interval T is the number of assumption the mean estimate of b for these rats (0.54 responses made in that interval (N) divided by T. There- resp/sec) was closer to the mean estimate of b for other fore, rats and epochs (0.42 resp/sec) than to the estimates of N = BT (A1) w for other rats and epochs (2.28 resp/sec). Brackney and colleagues (2011) considered δ, the shortest possible If T is the interval between trial onset and reinforce- IRT, a better estimation of motoric constraint than w. ment, then T may be partitioned into two periods: the Because of the low temporal resolution at which time between trial onset and the first response (latency, responses were recorded in the present study (9 Hz), δ or L)and thetimebetween thefirst response andthe could not be analyzed separately. It is likely, however, reinforced response (T - L). The latter may be further that variations in w between strains comprise variations portioned out into N - 1 inter-response times (IRTs). in δ, particularly at high values of w. The mean IRT is t =(T - L)/(N -1). Solvingfor T in the mean IRT equation and then substituting N with BT, List of Abbreviations ADHD: Attention deficit hyperactivity disorder; SHR: Spontaneously T = L +(BT − 1)t; (A2) hypertensive rat; WKY: Wistar Kyoto rat; WIS: Wistar rat; VI: Variable interval; IRT: Inter-response time; AICc: Corrected Akaike Information Criterion. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 14 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 19. van den Bergh FS, Bloemarts E, Chan JSW, Groenink L, Olivier B, Oosting RS: Acknowledgements and funding Spontaneously hypertensive rats do not predict symptoms of attention- The authors would like to thank Peter Killeen for helpful comments on deficit hyperactivity disorder. Pharmacol Biochem Behav 2006, 83:380-390. analyses and earlier drafts of the manuscript. We thank Jonathan Schiro and 20. Paulus M, Geyer M: Three independent factors characterize spontaneous Alison Moritz for help with figure editing. rat motor activity. Behav Brain Res 1993, 53:11-20. This project was supported by startup funds from the College of Liberal Arts 21. Ferster C, Skinner B: Schedules of Reinforcement. New York: Appleton- and Sciences, Arizona State University, to FS. Century-Crofts; 1957. 22. Hsieh Y, Yang C: Age-series characteristics of locomotor activities in Authors’ contributions spontaneously hypertensive rats: a comparison with the Wistar-Kyoto FS conceived and designed the study. JH and KH acquired and analyzed strain. Physiol Behav 2008, 93:777-782. data. JH, KH and FS interpreted data and drafted and revised the 23. Knardah S, Sagvolden T: Open-field behavior of spontaneously- manuscript. Parts of the manuscript served as Honors thesis for KH. All authors read and approved the final manuscript. hypertensive rats. Behav Neural Biol 1979, 27:187-200. 24. Gentsch C, Lichtsteiner M, Feer H: Open field and elevated plus-maze: A Competing interests behavioral comparison between spontaneously hypertensive (SHR) and The authors declare that they have no competing interests. Wistar-Kyoto (WKY) rats and the effects of chlordiazepoxide. Behav Brain Res 1987, 25:101-107. Received: 26 September 2011 Accepted: 26 January 2012 25. Sagvolden T, Hendley ED, Knardahl S: Behavior of hypertensive and Published: 26 January 2012 hyperactive rat strains: hyperactivity is not unitarily determined. Physiol Behav 1992, 52:49-57. 26. Myers M, Musty R, Hendley E: Attenuation of hyperactivity in the References spontaneously hypertensive rat by amphetamine. Behav Neural Biol 1982, 1. Froehlich T, Lanphear BP, Epstein JN, Barbaresi WJ, Katusic SK, Kahn RS: 34:42-54. Prevalence, recognition, and treatment of attention-deficit/hyperactivity 27. Danysz W, Plaźnik A, Pucilowski O, Plewako M, Obersztyn M, Kostowski W: disorder in a national sample of US children. Arch Pediatr Adolesc Med Behavioral studies in spontaneously hypertensive rats. Behav Neural Biol 2007, 161(9):857-864. 1983, 39:22-29. 2. Polanczyk G, de Lima, Horta B, Biederman J, Rohde L: The worldwide 28. Cierpial M, M McCarty R: Maternal effects on the development of prevalence of ADHD: a systematic review and metaregression analysis. spontaneous hypertension. Health Psychol 1988, 7:125-135. Am J Psychiatry 2007, 164:942-948. 29. Ferguson S, Cada A: A longitudinal study of short- and long-term activity 3. Kessler RC, Adler L, Barkley R, Biederman J, Conners K, Demler O, levels in male and female spontaneously hypertensive, Wistar-Kyoto, Faraone SV, Greenhill LL, Howes MJ, Secnik K, Spencer T, Ustun TB, and Sprague-Dawley rats. Behav Neurosci 2003, 117:271-282. Walters EE, Zaslavsky AM: The prevalence and correlates of adult ADHD in Amini B, Swann A, Dafny N: Differential locomotor responses in male rats the United States: results from the national comorbidity survey from three strains to acute methylphenidate. Int J Neurosci 2004, replication. Am J Psychiatry 2006, 163:716-723. 114:1063-1084. 4. Faraone SV, Sergeant J, Gillberg C, Biederman J: The worldwide prevalence 31. Sagvolden T, Pettersen M, Larsen M: Spontaneously hypertensive rates of ADHD: is it an American condition? World Psychiatry 2003, 2(2):104-113. (SHR) as a putative animal model of childhood hyperkinesis: SHR 5. American Psychiatric Association: Diagnostic and statistical manual of behavior compared to four other rat strains. Physiol Behav 1993, mental disorders. Washington, DC: Author;, 4 2000. 54(6):1047-1055. 6. Barkley RA: Behavioral inhibition, sustained attention, and executive 32. Sagvolden T, Metzger MA, Sagvolden G: Frequent rewards eliminates functions: constructing a unifying theory of ADHD. Psychol Bull 1997, differences in activity between hyperkinetic rats and controls. Behav 121:65-94. Neurol Biol 1993, 59:225-229. 7. Barry TD, Lyman RD, Klinger LG: Academic under-achievement and 33. Williams J, Sagvolden G, Taylor E, Sagvolden T: Dynamic behavioral attention-deficit/hyperactivity disorder: the negative impact of symptom changes in the Spontaneously Hypertensive Rat 1. Control by place, severity on school performance. J School Pscyhol 2002, 40(3):259-283. timing, and reinforcement rate. Behav Brain Res 2009, 198:273-282. 8. Spira EG, Fischel JE: The impact of preschool inattention, hyperactivity, 34. Johansen E, Sagvolden S, Kvande G: Effects of delayed reinforcers on the and impulsivity on social and academic development: a review. J Child behavior of an animal model of attention-deficit/hyperactivity disorder Psychol Psych 2005, 46:755-773. (ADHD). Behav Brain Res 2005, 162(1):47-61. 9. Loe IM, Feldmen HM: Academic and educational outcomes of children 35. Sagvolden T, Aase H, Zeiner P, Berger DF: Altered reinforcement with ADHD. Ambul Pediatr 2007, 7(1):82-90. mechanisms in ADHD. Behav Brain Res 1998, 33:387-398. 10. Sagvolden T, Russell VA, Aase H, Johansen EB, Farshbaf M: Rodent models 36. Douglas V, Parry P: Effects of reward on delayed reaction time task of attention-deficit/hyperactivity disorder. Biol Psychiatry 2005, performance in hyperactive children. J Abnorm Child Psych 1983, 57:1239-1247. 11:313-326. 11. Russell V: Neurobiology of animal models of attention-deficit 37. Aase H, Sagvolden T: Infrequent, but not frequent, reinforcers produce hyperactivity disorder. J Neurosci Methods 2007, 161:185-198. more variable responding and deficient sustained attention in young 12. Bari A, Robbins TW: Animal models of ADHD. Molecular and Functional children with attention-deficit/hyperactivity disorder (ADHD). J Child Models in Neuropsychiatry: Curr Top Behav Neurosci 2011, 7:149-185. Psychol Psych 2006, 47:457-471. 13. Sanabria F, Killeen PR: Evidence for impulsivity in the Spontaneously 38. Sagvolden T, Johansen EB, Aase H, Russell VA: A dynamic developmental Hypertensive Rat drawn from complementary response-withholding theory of attention-deficit/hyperactivity disorder (ADHD) predominantly tasks. Behav Brain Funct 2008, 4:7-24. hyperactive/impulsive and combined subtypes. Behav Brain Sci 2005, 14. Fox AT, Hand DJ, Reilly MP: Impulsive choice in a rodent model of 28:397-419. attention-deficit/hyperactivity disorder. Behav Brain Res 2008, 187:146-152. 39. de Villiers P, Herrnstein RJ: Toward a law of response strength. Psych Bull 15. Evenden J, Meyerson B: The behavior of Spontaneously Hypertensive and 1976, 83:1131-1153. Wistar Kyoto rats under a paced fixed consecutive number schedule of 40. Herrnstein RJ: On the law of effect. J Exp Anal Behav 1970, 13(2):243-266. reinforcement. Pharmacol Biochem Behav 1999, 63:71-82. 41. Blough DS: Interresponse time as a function of continuous variables: a 16. Berger DF, Sagvolden T: Sex differences in visual discrimination new method and some data. J Exp Anal Behav 1963, 6:237-246. behaviour in an animal model of attention deficit hyperactivity disorder. 42. Shull RL, Gaynor ST, Grimes JA: Response rate viewed as engagement Behav Brain Res 1998, 94:73-82. bouts: effects of relative reinforcement and schedule type. J Exp Anal 17. Sagvolden T: Behavioral validation of the spontaneously hypertensive rat Behav 2001, 75:247-274. (SHR) as an animal model of attention-deficit/hyperactivity disorder (AD/ 43. Shull RL, Grimes JA, Bennett JA: Bouts of responding: the relation HD). Neurosci Biobehav Rev 2000, 24:31-39. between bout rate and the rate of variable-interval reinforcement. J Exp 18. Alsop B: Problems with spontaneously hypertensive rats (SHR) as a Anal Behav 2004, 81:65-83. model of attention-deficit/hyperactivity disorder (AD/HD). J Neurosci Methods 2007, 162:42-48. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 15 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 44. Shull RL, Grimes JA: Bouts of responding from variable-interval 71. Johansen EB, Killeen PR, Sagvolden T: Behavioral variability, elimination of reinforcement of lever pressing by rats. J Exp Anal Behav 2003, responses, and delay-of-reinforcement gradients in SHR and WKY rats. 80:159-171. Behav Brain Funct 2007, 3:60. 45. Brackney RJ, Cheung TH, Neisewander JL, Sanabria F: The isolation of 72. Johansen EB, Sagvolden T: Slower extinction of responses maintained by motivational, motoric, and schedule effects on operant performance: a intra-cranial self-stimulation (ICSS) in an animal model of attention- modeling approach. J Exp Anal Behav 2011, 37:1-9. deficit/hyperactivity disorder (ADHD). Behav Brain Res 2005, 162:22-31. 46. Reed P: An experimental analysis of steady-state response rate 73. Sutherland KR, Alsop B, McNaughton N, Hyland BI, Tripp G, Wickens JR: components on variable ratio and variable interval schedules of Sensitivity to delay of reinforcement in two animal models of attention reinforcement. J Exp Psychol [Anim Behav] 2011, 39:27-35. deficit hyperactivity disorder (ADHD). Behav Brain Res 2009, 205:372-376. 47. Shull RL: Bouts, changeovers, and units of operant behavior. Eur J Behav 74. Sagvolden T, Aase H, Zeiner P, Berger D: Altered reinforcement Anal 2011, 12:49-72. mechanisms in attention-deficit/hyperactivity disorder. Behav Brain Res 48. Conover KL, Fulton S, Shizgal P: Operant tempo varies with reinforcement 1998, 94:61-71. rate: implications for measurement of reward efficacy. Behav Process 75. Johansen EB, Killeen PR, Russell VA, Tripp G, Wickens JR, Tannock R, 2001, 56:85-101. Williams J, Sagvolden T: Origins of altered reinforcement effects in ADHD. 49. Sagvolden T, Johansen E, Woien G, Walaas SI, Storm-Mathisen J, Behav Brain Funct 2009, 5:7. Bergerson LH, Hvalby O, Jensen V, Aase H, Russell V, Killeen PR, 76. Luman M, Tripp G, Scheres A: Identifying the neurobiology of altered DasBanerjee T, Middleton F, Faraone SV: The spontaneously hypertensive reinforcement sensitivity in ADHD: a review and research agenda. rat model of ADHD- The importance of selecting the appropriate Neurosci Biobehav Rev 2010, 34:744-754. reference strain. Neuropharmacology 2009, 57:619-626. 77. Taylor D, Lincoln AJ, Foster SL: Impaired behavior regulation under 50. Fleshler M, Hoffman HS: A progression for generating variable-interval conditions of concurrent variable schedules of reinforcement in children schedules. J Exp Anal Behav 1962, 5:529-530. with ADHD. J Attention Disord 2010, 13:358-368. 51. Spear LP, Brake SC: Periadolescence: Age-dependent behavior and 78. Hart EL, Lahey BB, Loeber R, Applegate B, Green SM, Frick PJ: pyschopharmacological responsivity in rats. Dev Psychobiol 2009, Developmental change in attention-deficit hyperactivity disorder in 16:83-111. boys: a four-year longitudinal study. J Abnorm Child Psych 1995, 52. Herrnstein RJ: Relative and absolute strength of responses as a function 23:729-749. of frequency of reinforcement. J Exp Anal Behav 1961, 4:267-272. 79. Faraone SV, Biederman J: What is the prevalence of adult ADHD? Results 53. Shull RL: The sensitivity of response rate to the rate of variable-interval of a population screen of 966 adults. J Attention Disord 2005, 9:384-391. reinforcement for pigeons and rats: a review. J Exp Anal Behav 2005, 80. Faraone S, Biederman J, Mick E: The age-dependent decline of attention- 84:99-110. deficit/hyperactivity disorder: a meta-analysis of follow-up studies. 54. Fantino E, Davison M: Choice: some quantitative relations. J Exp Anal Psychol Med 2006, 36:159-165. Behav 1983, 40(1):1-13. doi:10.1186/1744-9081-8-5 55. Bradshaw CM, Ruddle HV, Szabadi E: Relationship between response rate Cite this article as: Hill et al.: Characterizing operant hyperactivity in the reinforcement frequency in variable-interval schedules: III. the effect of Spontaneously Hypertensive Rat. Behavioral and Brain Functions 2012 8:5. d-amphetamine. J Exp Anal Behav 1981, 36(1):29-39. 56. Martens BK, Lochner DG, Kelly SQ: The effects of variable interval reinforcement on academic engagement: a demonstration of matching theory. J Exp Anal Behav 1992, 25(1):143-151. 57. Dallery J, Soto PL: Herrnstein’s hyperbolic matching equation and behavioral pharmacology: review and critique. Behav Pharmacol 2004, 15:443-459. 58. Burnham KP, Anderson DR: Model selection and multimodel inference: A practical information-theoretic approach New York NY, Springer-Verlag; 2002. 59. Myung I: Tutorial on maximum likelihood estimation. J Math Psychol 2003, 47:90-100. 60. Bowers MT, Hill J, Palya WL: Interresponse time structures in variable- ration and variable-interval schedules. J Exp Anal Behav 2008, 90:345-362. 61. Johnson JE, Pese EF, Newland MC: High-rate operant behavior in two mouse strains: a response-bout analysis. Behav Process 2009, 81:309-315. 62. Podlesnik CA, Jimenez-Gomez C, Ward RD, Shahan TA: Resistance to change of responding maintained by unsignaled delays to reinforcement: a response-bout analysis. J Exp Anal Behav 2006, 85:329-347. 63. Malott RW: The effects of prefeeding in plain and chained fixed ratio schedules of reinforcement. Psychon Sci 1966, 4:285-286. 64. Sidman M, Stebbins WC: Satiation effects under fixed-ratio schedules of reinforcement. J Comp Physiol Psychol 1954, 47:114-116. 65. Killeen P: Reinforcement frequency and contingency as factors in fixed- ratio behavior. J Exp Anal Behav 1969, 12(3):391-395. 66. Dickinson A, Watt A, Griffiths WJH: Free-operant acquisition with delayed Submit your next manuscript to BioMed Central reinforcement. Q J Exp Psychol-B 1992, 45:241-258. and take full advantage of: 67. Killeen PR, Fantino E: Unification of models for choice between delayed reinforcers. J Exp Anal Behav 1990, 53(1):189-200. • Convenient online submission 68. Killeen PR: Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid. Behav • Thorough peer review Process 2011, 87:57-63. • No space constraints or color figure charges 69. Lattal KA: Delayed reinforcement of operant behavior. J Exp Anal Behav • Immediate publication on acceptance 2010, 93:129-139. 70. Hand DJ, Fox AT, Reilly MP: Response acquisition with delayed • Inclusion in PubMed, CAS, Scopus and Google Scholar reinforcement in a rodent model of attention-deficit/hyperactivity • Research which is freely available for redistribution disorder (ADHD). Behav Brain Res 2006, 175:337-342. Submit your manuscript at www.biomedcentral.com/submit http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Behavioral and Brain Functions Springer Journals

Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat

Download PDF
15 pages

Loading next page...
 
/lp/springer-journals/characterizing-operant-hyperactivity-in-the-spontaneously-hypertensive-hNmFey4S98

References (160)

Publisher
Springer Journals
Copyright
Copyright © 2012 by Hill et al; licensee BioMed Central Ltd.
Subject
Biomedicine; Neurosciences; Neurology; Behavioral Therapy; Psychiatry
eISSN
1744-9081
DOI
10.1186/1744-9081-8-5
pmid
22277367
Publisher site
See Article on Publisher Site

Abstract

Background: Operant hyperactivity, the emission of reinforced responses at an inordinately high rate, has been reported in children with ADHD and in the Spontaneously Hypertensive Rat (SHR), the most widely studied animal model of ADHD. The SHR emits behavior at hyperactive levels, relative to a normoactive strain, only when such behavior is seldom reinforced. Because of its dependence on rate of reinforcement, operant hyperactivity appears to be driven primarily by incentive motivation, not motoric capacity. This claim was evaluated in the present study using a novel strategy, based on the organization of behavior in bouts of reinforced responses separated by pauses. Method: Male SHR, Wistar-Kyoto (WKY) and Wistar rats (WIS) were exposed each to a multiple variable-interval schedule of sucrose reinforcement (12, 24, 48, 96, and 192 s) between post-natal days (PND) 48 and 93. Responding in each schedule was examined in two epochs, PND 58-62 and 89-93. Parameters of response- reinforcement functions (Herrnstein’s hyperbola) and bout-organized behavior were estimated in each epoch. Results: SHR emitted higher response rates than WKY and WIS, but only when rate of reinforcement was low (fewer than 2 reinforcers per minute), and particularly in the second epoch. Estimates of Herrnstein’s hyperbola parameters suggested the primacy of motivational over motoric factors driving the response-rate differential. Across epochs and schedules, a more detailed analysis of response bouts by SHR revealed that these were shorter than those by WKY, but more frequent than those by WKY and WIS. Differences in bout length subsided between epochs, but differences in bout-initiation rate were exacerbated. These results were interpreted in light of robust evidence linking changes in bout-organization parameters and experimental manipulations of motivation and response-reinforcement contingency. Conclusions: Operant hyperactivity in SHR was confirmed. Although incentive motivation appears to play an important role in operant hyperactivity and motoric capacity cannot be ruled out as a factor, response-bout patterns suggest that operant hyperactivity is primarily driven by steeper delay-of-reinforcement gradients. Convergence of this conclusion with theoretical accounts of ADHD and with free-operant performance in children with ADHD supports the use of SHR as an animal model of ADHD. Keywords: ADHD, Spontaneously Hypertensive rat, hyperactivity, operant, variable interval, bout Background school, poor interpersonal relationships, and psychologi- Attention deficit hyperactivity disorder (ADHD) is the cal problems such as depression and anxiety, among most commonly diagnosed childhood psychiatric disor- others [6-9]. The Spontaneously Hypertensive Rat (SHR) is the der, affecting between 2% and 12% of grade school chil- dren, and around 4% of adults [1-4]. It is characterized most widely used animal model of ADHD [10-12]. Evi- by difficulties related to impulsivity, inattention, and dence suggests that SHR displays the three main beha- hyperactivity [5]. ADHD is associated with problems in vioral characteristics of ADHD: impulsivity [13-15], inattention [16], and hyperactivity [17]. Nonetheless, the reliability of some of this evidence and the validational * Correspondence: Federico.Sanabria@asu.edu support it provides has been disputed [18,19]. Sanabria Department of Psychology, Arizona State University, P.O. Box 871104, Tempe, Arizona 85287-1104, USA © 2012 Hill et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 2 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 and Killeen [13] addressed the inconsistency of the evi- contribute to operant hyperactivity. The purpose of the dence regarding response inhibition deficits in SHR. present study is to advance the identification of such They concluded that, without a model of response inhi- processes by examining SHR performance at a high bition and appropriate procedures for estimating model level of detail. parameters, claims about impulsivity in SHR are unlikely Inferences on motor and motivational processes have to converge. The purpose of this paper is to extend this been drawn from performance in variable interval (VI) schedules of reinforcement. In these schedules, the first reasoning to another symptom of ADHD that is pre- response following an unsignaled interval of variable sumably expressed in the SHR: hyperactivity. duration is reinforced. Inferences are based on para- Hyperactivity in SHR has been assessed using the open-field method and the operant-conditioning meters of models fit to average response rates in VI method. The open-field method consists of measuring schedules [39,40]. Not all responses in VI schedules, the amount of locomotor activity (typically in the form however, are functionally equivalent, so averaging all of infrared beam breaks) in an enclosure [20]. Because responses in a session may neglect useful information activity measured by this method does not yield pro- [41]. In fact, the distribution of inter-response times grammed consequences, we refer to it as spontaneous (IRTs) provides additional information about the multi- activity. The operant-conditioning method consists of ple sources of variance in VI performance [42-47]. Fig- measuring the rate of emission of a target response ure 1 illustrates the IRT model used in the present (typically lever pressing), where the target response study, which we call the bout-and-pause model. Each occasionally produces a reinforcer (typically food). vertical line represents a response; the spaces between Because activity measured by this method operates on a vertical lines represent the IRTs. The critical assumption specific feature of the environment and yields a pro- of this model is that operant responding occurs in bouts grammed consequence, we refer to it as operant activity. separated by relatively long pauses [40,41,48]. Thus, Operant activity is often observed under interval sche- increased responding in the SHR at low rates of reinfor- dules of reinforcement, in which only the first target cement may be due to (1) faster responding within response following a programmed interval is reinforced. bouts, (2) longer bouts, or (3) shorter pauses between Interval schedules maintain an approximately constant bouts. The purpose of this study was to replicate past rate of reinforcement regardless of response rate [21], results that show that SHR hyperactivity is constrained thus isolating changes in rate of reinforcement from to low rates of reinforcement, and to characterize oper- ant hyperactivity in SHR in terms of bout-and-pause changes in activity. parameters. It is not clear that the SHR displays more spontaneous activity than control strains. Whereas some research has demonstrated spontaneous hyperactivity in the SHR [22-27], other research has shown this effect only at cer- tain ages [19,28], and still other research has not shown such an effect [29-31]. In contrast, operant hyperactivity is well demonstrated in the SHR [16,25,31-34]. Under interval schedules of reinforcement, the SHR typically responds at significantly higher rates than control strains [16,17]. An analogous difference has been observed between children with and without ADHD [35]. Performance under varying rates of reinforcement has been informative of the nature of operant hyperactivity in the SHR. Response rates in the SHR and control strains covary with rate of reinforcement, but the SHR responds at abnormally higher rates only when rate of Figure 1 Schematic timeline depicting the bout-and-pause reinforcement is low [32,33]. These researchers showed model of free-operant performance. A trial starts at the left-end of the timeline and progresses to the right. The time between trial that maximal responding was about equal for SHR and onset and the first response (vertical line labeled “Response 1”)is Wistar-Kyoto rats (WKY, which typically serves as con- the latency for that trial. The time between two consecutive trol strain), suggesting that superior motor ability alone responses is an inter-response time, or IRT. There are two types of cannot explain hyperactivity in the SHR. High rates of IRTs: within-bout (short) and between-bout (long). Thicker vertical reinforcement have also been shown to normalize the lines are bout-initiation responses; thinner lines are within-bout responses. The length of a bout is the number of responses operant performance of children with ADHD [36,37]. between latency and the first between-bout IRT, or between two These results suggest that non-motoric processes, such between-bout IRTs. as differences in responsiveness to incentives [38], may Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 3 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 lever presses when a force of about 0.2N was applied to Methods the lever. Responses were recorded with a 110-ms reso- Subjects lution. The house light was not turned on for experi- Eighteen male rats, 6 each of three different strains were mental sessions. A speaker located at the top of the side obtained from Charles River Laboratories (US) (Sponta- wall opposite to the food receptacle emitted 75 dB tones neously Hypertensive Rats, SHR; Wistar rats, WIS) and from a generator (ENV-223). Harlan Laboratories (US) (Wistar-Kyoto rats, WKY). The substrain of WKY bred by Harlan Laboratories has Procedure been shown to be the most appropriate normoactive control strain for the SHR [49]. Hopper training and autoshaping Experimental sessions were conducted daily at approxi- Rats arrived to the laboratory on post-natal day (PND) mately the same time of day for each rat. Hopper train- 24-25. Hopper training commenced on PND 39. Rats ing consisted of presenting a food pellet every 15 s on were pair-housed within their strains in a colony room average. After two days, rats were eating consistently with a 12:12 hour light:dark cycle; experiments were from the hopper and autoshaping of lever pressing conducted during the dark cycle. Rats were maintained started. Autoshaping consisted of extending a randomly at 85% of their free-feeding weights based on a logistic selected lever (left or right) for 10 s, every 30 s; a food function fitted to the growth curves provided by bree- pellet followed only right lever retractions. After 9 days ders. All rats were handled for a minimum of 2 min/day rats were consistently pressing the right lever each time by the researchers in the days preceding hopper train- it was extended. ing. Animals were weighed every morning, and fed a Operant Task supplementary amount of rodent chow every evening, at The first five minutes of each session served as an accli- least 12 hr before the following experimental session. At mation period, during which the houselight was off and the beginning of operant training the mean weights for the levers were retracted. After the acclimation period, the SHR, WKY, and WIS were 118, 127, and 190 g, the right lever was extended into the chamber, and a respectively. Home cages were always equipped with multiple variable interval (VI) schedule was in effect. water bottles. All handling procedures in the present One of five VI schedules (VI 12, 24, 48, 96, or 192 s) study were maintained according to the guidelines of was randomly selected. Schedules were implemented on the National Institute for Health, which were approved each trial by selecting without replacement from an 8- by the Arizona State University Institutional Animal item Fleschler-Hoffman distribution of intervals [50]; Care and Use Committee. the mean of the distribution was the nominal VI Apparatus requirement. Responses that occurred during the inter- val were recorded but had no programmed effect. Once All experimental sessions were conducted in 6 MED the selected interval time elapsed, the first lever press Associates modular test chambers (305 mm × 241 mm resulted in the delivery of one food pellet into the feed- × 210 mm). Each sound- and light-attenuating box con- ing aperture, which served as reinforcement. After each tained a ventilating fan. The fan provided a masking pellet delivery, the lever was retracted, a 5-s inter-trial noise of about 60 dB. The bottom of each box was lined interval (ITI) ensued, then the lever was extended again with a catch pan full of sanitary chip litter, and the floor and another interval was selected from the same VI dis- of each chamber had thin metal bars. The front and tribution. When an 8-item distribution was exhausted, back walls and the ceiling were made of clear polycarbo- the ITI was 20 s and another VI schedule was selected. nate; the front wall also served as a door. The food The five schedules of reinforcement were signaled by receptacle was attached to a square aluminum aperture one of five tones. Each tone (3-12 kHz) was presented (51 mm sides, 15 mm above the chamber floor), cen- on a unique on:off cycle (200-1000 ms) for the duration tered on the side wall against which the chamber door of the schedule. Sessions ended when every schedule was latched. Activation of the food dispenser released was implemented once, or after 70 minutes, whichever one 45-mg food pellet (Dustless Precision Pellets , happened first. Fifty-four daily sessions were conducted, Rodent Grain-Based Diet, Bio-Serv, Frenchtown, NJ). 7 days/week. Although the study only involved use of the right lever Measures (closest to the door of the chamber), two retractable The first analysis was based on two measures: response levers (MED associates, ENV-112CM) were on either rate and reinforcement rate. Response rate was com- side of the food hopper. The inside edge of each lever puted for each VI schedule as the number of responses was 8 mm from the closest vertical edge of the recepta- emitted while the schedule was effective, divided by the cle. A Med-PC interface connected to a PC computer ran Med-PC IV software. This computer recorded time the schedule was in effect (excluding ITIs). Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 4 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Reinforcement rate was computed for each VI as the number of reinforcers collected at each schedule, divided by the time the schedule was in effect (excluding ITIs). Response rates were measured in two epochs, PND 58-62 (epoch 1) and PND 89-93 (epoch 2), follow- ing approximately 10 and 40 sessions of VI training, respectively. Epoch 1 corresponds to a conservative esti- mate of early adulthood, but possibly captures late ado- lescence. Epoch 2 corresponds to adulthood [51]. Herrnstein’s (1970) hyperbola parameters were esti- mated on the basis of response rates (explained in Results section) [40]. In a subsequent analysis, response rates were further analyzed by separating response latencies from inter- response times (IRTs). The distinction between latencies and IRTs is depicted in Figure 1. Latencies were the intervals between trial onset (lever extension) and the first lever press in that trial. IRTs were the intervals between consecutive lever presses within the same trial. Latencies were classified in two groups: the first latency in each VI (Latency 1), and all subsequent latencies within the same VI (Latencies 2-8). This classification took into account that, within each VI schedule, the duration of the first interval to reinforcement could only be cued by the discriminative tone, whereas the duration of subsequent intervals could also be cued by the dura- tion of the preceding intervals. Median Latencies 1 and 2-8 were computed separately for each rat within each Figure 2 Mean (± SEM) response rates of each strain (SHR: unfilled squares; WKY: filled circles; WIS: filled triangles) as a VI schedule and epoch, and then averaged within strain function of mean rate of reinforcement, in two epochs: PND (mean median latencies). Estimates of bout-initiation 58-62 (epoch 1; top panel) and PND 89-93 (epoch 2; bottom rates, within-bout response rates, and bout length-the panel). Response rate increased with rate of reinforcement in all parameters of the bout-and-pause model-were based on strains and epochs. SHR response rates were higher than those of the distribution of IRTs (explained in Results section). WKY and WIS when reinforcement was delivered less than twice per minute. At higher rates of reinforcement, WIS response rates were lower than those of SHR and WKY. Curves through the data are Results traces of Herrnstein’s hyperbola (Equation 1). Figure 2 shows mean (± SEM) response rates of each strain as a function of rate of reinforcement in each epoch. Response rates of SHR are indicated by unfilled more noticeable at higher rates of reinforcement, and squares, WKY by filled circles, and WIS by filled trian- forWIS therewas virtually no changein responserate gles. Visual inspection of Figure 2 reveals a positive cor- with age. To characterize these patterns of response relation between response rate and rate of rate, we estimated the parameters of Herrnstein’s (1970) reinforcement in all strains. Differences in response hyperbola and compared them across strains [40]. rates between strains and across rates of reinforcement are visible in both epochs. When rates of reinforcement Herrnstein’s hyperbola were low (fewer than 2 responses per minute), SHR Herrnstein (1970) extended the Matching Law [52] to responded at a higher rate than other strains. At higher describe the relation between response rate (B)and rate rates of reinforcement, SHR and WKY response rates of reinforcement (R) on a single operandum. Herrn- converged, and WIS response rates remained low (40-50 stein’s rationale was that all the responses other than responses per minute). These patterns of response rate the target response are reinforced at an unknown rate. across strains and schedules were visible in epoch 1 and Such rate, however, may be estimated if it is assumed were magnified in epoch 2. SHR response rate increased that (a) the ratio of two response rates matches the ratio with age regardless of rate of reinforcement, whereas for of the corresponding reinforcement rates (Matching WKY age-dependent increases in response rate were Law), and (b) the target response rate and the non- Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 5 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 target response rate add to a constant k. Under such where n is the number of observations (n =5sche- assumptions, dules × 18 rats = 90 observations in each epoch), RSS is the minimized residual sum of squares obtained from kR fitting a hypothesis to the data, and c is the number of B = , (1) R + R free parameters in the hypothesis. c can also be com- puted as the degrees of freedom of the estimates of the where R is the estimated rate of reinforcement pro- overall means of k and R plus 1 parameter for error vided by non-target responses. When reinforcement is variance, i.e., 2 parameters × 18 rats - the number of programmed on VI schedules, R typically falls only constraints + 1. In the preceding example, c =36-3+ slightly below programmed reinforcement rates and 1 = 34 free parameters. thus serves as the independent measure; B is the depen- Note that AICc increases with RSS and with c;there- dent measure; k and R are free parameters. Equation 1 fore smaller AICc are indicative of close fit to the data predicts that responding increases at a negative pace as and parsimony. Hypotheses with smaller AICc were reinforcement increases, with asymptote k. R is the rate favored over those with higher AICc. ΔAICc was com- of reinforcement at which response rate reaches half of puted as the difference between each the AICc of its asymptote (i.e., when R = R ,B = k /2). hypothesis i and the lowest AICc among all hypotheses Following Herrnstein’s (1970) rationale [40], k is often (ΔAICc =AICc -AICc ). The hypothesis with few- i i MIN interpreted as a maximum limit on motoric perfor- est free parameters among those with ΔAICc < 4 was mance, influenced only by response characteristics; R is selected as the best description of the data. This selec- interpreted as indexing motivation for the reinforcer, tion was conducted separately for the 2 epochs in which influenced only by reinforcer characteristics [39]. A data were collected. large body of evidence supports Equation 1 as an accu- Table 1 shows the 5 hypotheses with the lowest rate characterization of response-reinforcement func- ΔAICc in each epoch. The selected hypothesis for epoch tions like those in Figure 2[53-56]. The empirical 1 assumes that R = R , and all other parameters eWIS eSHR support for motoric/motivational interpretations is, varied between strains. For epoch 2, the selected however, somewhat mixed [57]. hypothesis assumes different parameters for each strain. Parameters of Equation 1 were estimated by fitting Figure 3 shows the mean estimates of k and R for each Equation 1 to the data of each individual animal, in epoch based on the selected hypotheses. It was inferred each epoch, using the method of least squares. Para- that k >k >k in both epochs, which indicates WKY SHR WIS meters k and R were assumed constant across values of that WKY had the highest asymptotic response rates, R, but could vary between rats, thus yielding 2 × 18 = followed by the SHR and then WIS. Mean k estimates 36 model parameters. Comparisons were conducted increased across epochs for all strains. In epoch 1, between mean estimates of each strain, henceforth R >R = R ;atepoch 2, R >R >R . eWKY eWIS eSHR eWKY eWIS eSHR referred to by the parameter and strain abbreviation: k , k , k , R , R ,and R . The curves SHR WKY WIS eSHR eWKY eWIS Table 1 Hypotheses of VI performance with lowest ΔAICc. in Figure 2 are traces of Equation 1 using the mean esti- Hypothesis c RSS ΔAICc mates of k and Re for each strain. Various constraints were imposed on model para- Epoch 1 (PND 58-62) meters to draw inferences on between-strain differences. R = R 36 306.78 0.00 eSHR eWIS These constraints consisted of holding constant the None 37 306.12 5.49 mean estimate of either model parameter across all, k =k 36 458.24 36.11 WIS WKY some, or none of the strains. Each particular combina- k = k , R = R , 35 496.05 37.78 SHR WIS eWIS eWKY tion of constraints constituted a hypothesis. Thus, for k = k 36 495.73 43.19 SHR WIS example, k ≠ k = k , R = R = R is Epoch 2 (PND 89-93) SHR WKY WIS eSHR eWKY eWIS the hypothesis that mean k varied between SHR and None 37 581.35 0.00 WKY, but not between WKY and WIS, and mean R R = R 36 738.84 15.89 eWIS eWKY did not vary between strains. There were 15 possible k = k , R =R , 35 837.20 21.67 SHR WIS eWIS eWKY constraint combinations. k = k 36 807.35 23.87 SHR WIS Hypothesis testing was conducted separately in each R = R 36 904.42 34.09 eSHR eWIS epoch, using the corrected Akaike Information Criteria Note. The label of each hypothesis stipulates the constraints on mean parameter estimates. c is the number of free parameters in each hypothesis; (AICc) [58], RSS is the residual sum of squares from model fitting. For all hypotheses, the number of observations was n = 90. See Equation 2 and text for computation 2nc of ΔAICc. Hypotheses are arranged according to ΔAICc within each epoch. (2) AICc = n ln RSS/n + n − c − 1 Hypotheses with ΔAIC = 0 were selected for parameter estimation. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 6 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 select an unintelligible model [58]. Conventional approaches, such as null-hypothesis testing, are not designed for this task: falsifying the null hypothesis that a particular component did not vary between strains in one or more schedules provides little information on the contribution of that component to differences in response rate. Therefore, the analysis presented here is qualitative; inferences drawn from this analysis should be taken as exploratory and provisional, pending empiri- cal verification. Latencies Figure 4 shows mean (± SEM) median Latency 1 and Latencies 2-8 for each strain in each epoch, as a func- tion of rate of reinforcement. Latency 1 (left panels) did not vary systematically with rate of reinforcement in Figure 3 Mean (± SEM) estimates of Herrnstein’shyperbola either epoch or across strains in epoch 1. In epoch 2, parameters k (asymptotic target response rate; left panels) and R (rate of reinforcement of non-target behavior; right panels) mean median Latency 1 was longer for WKY than for for each strain in epochs 1 and 2 (top and bottom panels, the other strains, regardless of rate of reinforcement. respectively). Estimates are based on hypotheses selected The right panels of Figure 4 and their insets show that according to AICc (Table 1). Estimates of k for SHR were Latencies 2-8 declined with rate of reinforcement. In intermediate relative to other strains. Estimates of R for SHR were epoch 1, Latencies 2-8 were mostly undistinguishable low relative to other strains, and approximately constant across epochs. between strains, with the possible exception of the longer latencies of WKY at the lowest rate of reinforce- ment. In epoch 2, median Latencies 2-8 of WIS were R does not appear to change across epochs, whereas eSHR longer on average, but also more variable across rats, R and R increased. eWKY eWIS than those of SHR and WKY. Also in this epoch, when Inferences from Herrnstein’s hyperbola parameters rate of reinforcement was less than 1 per minute, mean suggest that instrumental overactivity could be attribu- median Latencies 2-8 were about 1 s shorter for SHR ted to a higher motivation for the reinforcer, which did than WKY. The slopes of rescaled Latencies 2-8 (each not decline over nearly 30 days that separated the two median latency was divided by the median latency in VI assessment epochs, and not to differences in motoric 12 s, then logged, base 2), shown in the insets, reveal a capacity. This analysis, however, was based on average within-subject sensitivity of Latencies 2-8 to rate of rein- response rates in each VI schedule, which conflate two forcement in both epochs. This sensitivity was more types of intervals within the denominator: response pronounced in WKY than in the other strains. latencies and inter-response times (IRTs). Because rodent VI performance is typically organized in bouts Inter-response times (IRTs): Bout-and-pause model [42], IRTs may be further disaggregated into between- To account for the distribution of IRTs in each sche- bout and within-bout IRTs. Latencies, between- and dule, response rate in each VI schedule, excluding laten- within-bout IRTs may each depend on a distinct set of cies, was disaggregated into bout-initiation rate (the variables [45], which may further inform the sources of reciprocal of the mean IRT separating response bouts) SHR overactivity. and within-bout response rate (the reciprocal of the In the next two sections we examine the components mean IRT within bouts). This disaggregation consisted of response rate in SHR, WKY, and WIS. This analysis of estimating the parameters of a bi-exponential density is aimed at identifying candidate components that may function by fitting it to the distribution of IRTs in each account for the differences in response rate between VI schedule. The density function is SHR and WKY selectively at low rates of reinforcement, −w(t−0.11) −b(t−0.11) and between SHR and WIS at all rates of reinforcement. p(IRT = t)= pwe +(1 − p)be , b ≤ w ≤ 9s;0 ≤ p ≤ 1 (3) An AIC-basedanalysisappearsto bebestsuitedto where p is the proportion of IRTs within bouts; 1/(1 - address this goal, because the relation between response p) is the mean bout length, measured in lever presses. w rate and its components is not linear (see Appendix). is theresponseratewithin bouts; b is the rate at which Without an apriori selection of hypotheses, however, bouts are initiated. Because responses take a minimum the combinatorial of parameters and factors implies a time to be produced, that minimum time (the shortest computationally intractable analysis that may ultimately Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 7 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Figure 4 Mean (± SEM) median Latency 1 (first latency within each VI schedule; left panels) and Latency 2-8 (right panels) as a function of mean rate of reinforcement, for each strain in epochs 1 and 2 (top and bottom panels, respectively). The insets of the right panels are mean rescaled latencies: each individual median Latency 2-8 in each schedule was divided by the median Latency 2-8 in VI 12 s of the same rat, and then logged (base 2). Thus, a rescaled latency of 3 indicates that the median latency in that VI (excluding Latency 1) was 2 = 8 times longer than in VI 12 s. For all strains, Latency 1 did not vary systematically with rate of reinforcement, whereas Latencies 2-8 were shorter with higher rates of reinforcement. SHR latencies were generally undistinguishable from those of the other strains, with the possible exception of the shorter SHR Latencies 2-8 when reinforcement was delivered less than once per minute. possible IRT) must be subtracted from the duration t of previously to illustrate differences in pause and bout every IRT [45]. The minimum IRT recorded for every responding in both rats and pigeons [42,60]. Often, rat was 0.11 s, which was the resolution at which these plots take on a “broken-stick” appearance with a responses were recorded. Therefore, 0.11 s were sub- steeply declining initial left limb and a more gradually tracted from t in the exponents of Equation 3, and w declining right limb. A long initial limb on the leftmost was constrained to be less than or equal to 1/0.11 ≈ 9 side of the graphs indicates a high proportion of within- responses per second. bout responses, p. The slope of the left limb is the Parameters of Equation 3 were estimated for each within-bout rate of responding, w; the slope of the right individual rat in each epoch using the method of maxi- limb is the rate of bout initiation, b. The curves in Fig- mum likelihood [59]. Figure 5 shows semi-log survival ure 5 show that Equation 3 provided a good fit of the plots of IRTs in each schedule and epoch, averaged data, although the broken-stick pattern was most clearly within each strain. These plots have been used visible in WKY at low rates of reinforcement. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 8 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Figure 5 Semi-log survival plots showing the mean proportion of IRTs greater than t in each schedule (symbols), strain (columns), and epoch (rows). Proportions were calculated for each rat in bins that contain, each, 1% of the IRTs; binned proportions were then averaged over rats. Curves through the data are the mean traces of the bout-and-pause model (Equation 3), drawn using maximally likely individual estimates. SHR survival functions (left panels) were steeper and more linear than those of WKY (center panels), indicating, respectively, shorter IRTs (higher response rate) and responses organized in less distinct bouts. Mean (± SEM) bout-and-pause parameter estimates rate of reinforcement. At the lowest rate of reinforce- for each strain at each VI schedule and epoch are ment, w =0.80and w = 1.17 responses per sec- WKY WIS shown in Figure 6. To compute the mean estimates of ond; at the highest rate of reinforcement, w =1.04 WKY w and b, individual estimates were weighed by p and (1 and w = 1.31 responses per second. These trends WIS - p), respectively, because confidence on w and b esti- were dwarfed by the large between-subject and mates co-varies with these weights. The top panels of between-schedule variability in estimates of w . More- SHR Figure 6 show the mean estimates of p, w and b in over, in every schedule, w >w >w (mean SHR WIS WKY epoch 1; the bottom panels show estimates in epoch 2. across schedules = 2.09, 1.24, and 0.96 responses per Mean bout-and-pause parameter estimates are labeled second, respectively). It is important to note, however, in the same way as Herrnstein’s hyperbola estimates (e. that estimates of w and w were based on 2-3 rats SHR WIS g., p , p , p ). of each strain, because p =0formost of theserats in WKY SHR WIS Estimates of the proportion of within-bout IRTs (p)in most schedules. epoch 1 are shown in the top-left panel of Figure 6. Estimates of bout initiation response rate (b)inthe Estimates of p were substantially higher (mean first epoch are shown in the top-right panel of Figure 6. WKY across schedules = .90) than those of p (.17) and Estimates of b and b systematically increased with SHR SHR WIS p (.26). This means that WKY produced substantially rate of reinforcement. At the lowest rate of reinforce- WIS longer bouts than SHR and WIS. Moreover, whereas ment, b = 0.36 and b = 0.32 responses per second; SHR WIS p and p were relatively constant across rates of at the highest rate of reinforcement, b =0.85and SHR WIS SHR reinforcement, p increased with higher rates of rein- b = 0.63 responses per second. Estimates of b var- WKY WIS WKY forcement, from .80 at the lowest rate to .99 at the high- ied as an inverted-U function of rate of reinforcement, est rate. peaking at the second highest rate of reinforcement Estimates of within-bout response rate (w) in epoch 1 (0.23 responses per second). In every schedule, b SHR areshown in thetop-middlepanel of Figure6. Esti- >b >b (mean across schedules = 0.61, 0.49, and WIS WKY mates of w and w increased only slightly with 0.15 responses per second, respectively). WKY WIS Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 9 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 Figure 6 Mean (± SEM) estimates of bout-and-pause parameters (Equation 3) as a function of mean rate of reinforcement, for each strain (SHR: squares; WKY: circles; WIS: triangles) in epochs 1 and 2 (top and bottom panels, respectively). In all epochs and schedules, SHR emitted shorter but more frequent bouts (low p in left panel, high b in right panel) than WKY. During epoch 1 SHR emitted faster within- bout responses (high w in top-center panel), but estimates of w were only possible for 2-3 out of 6 SHR rats; estimates of w were possible for all 6 WKY. The bottom panels of the Figure 6 show the mean (± Estimates of b in the second epoch are shown in the SEM) estimates of p, w and b in epoch 2. The bottom- bottom-right panel of Figure 6. As in the preceding left panel of Figure 6 shows that, similar to those in the epoch, b >b ≥ b in every VI schedule, and SHR WIS WKY preceding epoch, estimates of p in epoch 2 were estimates of b also increased with rates of reinforce- WKY very high and increased further with rate of reinforce- ment, including b . Estimates of b and b WKY SHR WKY ment. Estimates of p and p increased between increased between epochs in every VI schedule; b SHR WIS WIS epochs in every VI schedule. The increase was particu- remained relatively unchanged. larly noticeable in p ; on the average, estimates of SHR p more than tripled between epochs. Like in the pre- Discussion SHR ceding epoch, however, there were no trends in p Operant hyperactivity was observed in SHR, particularly SHR and p across VI schedules comparable to those of during adulthood (epoch 2, PND 89-93), but only at low WIS p . rates of reinforcement (less than 2 and 4 reinforcers per WKY Estimates of w in epoch 2 are shown in the bottom- minute on PND 58-62 and 89-93, respectively). Esti- middle panel of Figure 6. Estimates of w and w mates of Herrnstein’s hyperbola parameters (k, R ;see WKY WIS e increased between epochs in every schedule, but only Equation 1) suggest that operant hyperactivity in SHR is w preserved its positive correlation with rate of rein- not due to enhanced motor capacity relative to WKY WKY forcement. Estimates of w remained relatively high, (the converse is most likely the case: k >k ). SHR WKY SHR particularly when rate of reinforcement was low. Instead, highly valued activities-such as searching for Between-subject variance in individual estimates of w food-are less likely to be displaced by less valued, com- SHR increased substantially between epochs, further dwarfing peting activities in SHR than in WKY (R <R ). eSHR eWKY any differences between strains. The increase in This finding is consistent with the notion that frequent between-subject variance was due to 2 SHR with unde- reinforcement normalizes free-operant ADHD perfor- termined w in epoch 1, whose individual w estimates, mance [36,37]. averaged over VI schedules in epoch 2, were 5.50 and Differences between WKY and SHR in response rate 8.89 responses per second. Such high estimates were and in estimates of Herrnstein’s hyperbola parameters not obtained for any other rat of any strain. replicate prior findings in adult rats [32,33], and Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 10 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 generalize them, to a limited extent, to younger rats. the other 2 strains, particularly in epoch 1. SHR bouts Such generalization suggests that the inferences drawn became longer in adulthood, but were still systematically from the more detailed analysis, based on bout-and- shorter than those of WKY, regardless of schedule. The pause parameter estimates, are not idiosyncratic to the length and density of WKY bouts, unlike those of other present data, but reflect a more general phenomenon. strains, increased with increasing rate of reinforcement, Unlike Herrnstein’s hyperbola, the bout-and-pause which may explain why differences in overall response model does not assume the functional equivalence of all rate between SHR and WKY are confined to low rates operant responses. Instead, the bout-and-pause model of reinforcement. In epoch 1, WIS parameters were gen- erally intermediate to those of SHR and WKY across supports an analysis based on latency and IRT statistics that separates responses into more meaningful func- parameters; in epoch 2, WIS maintained relatively short tional categories, which we examine next. bouts that contained few responses. The bout-and-pause analysis thus identifies the higher Latencies frequency of bouts in SHR as the main source of hyper- Latencies shown in Figure 4 suggest that tones did not activity, and higher within-bout rate as a possible sec- support the discrimination between schedules of reinfor- ondary source. Rate of bout initiation is particularly cement. Instead, it appears that latencies were updated sensitive to motivational manipulations, increasing as a according to preceding intervals to reinforcement. Such function of reinforcement deprivation and availability in a process is evinced in the steeper Latency 2-8 curves rats [42-45], mice [61], and pigeons [62]. This correla- (right panels) relative to Latency 1 curves (left panel). tion suggests that SHR hyperactivity is caused by a When the interval to reinforcement was not cued by hypermotivation to incentives. Such inference is consis- prior intervals (Latency 1), WKY took longer to emit tent with latency patterns in adulthood, to the extent the first response, relative to SHR and WIS, but only in that latencies are indicative of motivation [63,64], and adulthood. In subsequent intervals (Latencies 2-8), WKY with inferences drawn from Herrnstein’s hyperbola para- latencies became particularly sensitive to rate of reinfor- meters, both here and in prior studies [32,33]. cement, especially in adulthood. Note that, in adulthood, the pattern of Latencies 2-8 (bottom-right panel) is a A delay-of-reinforcement-gradient hypothesis vertically-flipped analogue of response rates (Figure 2, Brackney and colleagues [45], however, caution against a bottom panel). This indicates that adult Latencies 2-8 straightforward interpretation of changes in rate of bout became either (1) an important determinant of response initiation in terms of incentive motivation. Based on the rate,or(2) sensitivetowhatever factors determined performance of Sprague-Dawley rats, they concluded response rate. Alternative (1) does not appear to be the that changes in bout initiation rate alone may be inter- case: based on mean adult performance of each strain, preted as changes in incentive motivation, but when for every latency there were about 11 - 20 IRTs in VI 12 such changes are accompanied by changes in other s, and up to 58 - 150 IRTs in VI 192 s. That is, latencies parameters, they may reflect non-motivational processes contributed between 0.67% and 8% of the response rate that only indirectly impact motivation. For instance, a denominator. Although latencies appear to reflect pat- tandem ratio requirement at the end of the VI lengthens terns of hyperactivity in SHR, particularly in adulthood, bouts and increases the number of responses within they cannot be the main source of these patterns. That them, but also reduces the frequency of bouts [42-45]. source is, therefore, most likely to be identified in the The latter effect cannot be accounted for by a reduction distribution of IRTs. in incentive motivation, because the tandem require- ment does not change the rate of reinforcement sub- IRTs stantially. Instead, Brackney and colleagues suggested The bout-and-pause model assumes that responses are that the tandem requirement favors the reinforcement organized in two separate categories: responses that of long response bouts [65], concomitantly reducing the initiate bouts and responses emitted within bouts. An temporal contiguity between bout initiations and rein- analysis based on bout-and-pause premises suggests that forcement. It is hypothesized that reduced initiation- SHR hyperactivity reflects a high rate of bout initiations reinforcement contiguity results in less effective reinfor- (higher b) in SHR relative to WKY (Figure 6, right cement of bout initiation and a consequent reduction in panels), even though SHR response bouts were shorter its rate. (lower p; Figure 6, left panels). Because of the short Brackney and colleagues’ [45] account of how a tan- length of their bouts, estimates of SHR within-bout dem requirement reduces bout initiation rate implies response rate (w) were not reliable. Nonetheless, the that the effectiveness of reinforcement declines with the performance of those SHR with p > 0 suggests that SHR temporal distance between the reinforced response (bout initiation) and the reinforcing event (food) [66,67]. within-bout response rates were higher than those of Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 11 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 The slope of this decline in reinforcement effectiveness children with ADHD [77]. The critical evidence was col- is the delay-of-reinforcement gradient [68,69]. The effect lected using concurrent independent VI schedules of of flattening the delay-of-reinforcement gradient on reinforcement, where two schedules, like those used in operant performance should be similar to the effect of the present study, were simultaneously in effect In this imposing a tandem requirement: It should facilitate the context, Shull [47] has argued that the length of a “visit” reinforcement of long bouts while reducing bout fre- to either schedule is functionally equivalent to the quency. Compared to the initiation of short bouts sup- length of bouts in a single-schedule design (following Herrnstein’s rationale, single-schedule designs may be ported by steeper gradients, the initiation of long bouts thought of as concurrent-schedule designs where one supported by flatter gradients should be less frequent because flatter gradients envelop more competing schedule is implicit [40]). Taylor, Lincoln and Foster responses (between-bout activities, within-bout [77] reported that children with ADHD switch more responses) than steeper gradients. These intuitions are between concurrent VI schedules, and thus produce diagrammed in Figure 7. shorter visits, than non-ADHD controls, as long as Compared to SHR, WKY displayed long-but-infre- switching between schedules is not penalized with a quent response bouts (high p and low b in Figure 6) in changeover delay. The converging patterns of free-oper- both assessment epochs. This pattern suggests that one ant performance in SHR and in children with ADHD important source of SHR hyperactivity is the steepness suggest that (1) the emission of short free-operant bouts of its delay-of-reinforcement gradient. This hypothesis is may be a diagnostic feature of the behavioral phenotype consistent with prior SHR data [34,38,70-73] and with of ADHD, revealing a deeper deficit in learning observations of children with ADHD [74]. The charac- response-reinforcement contingencies, and that (2) the terization of ADHD in terms of steeper delay-of-reinfor- SHR models these attributes of ADHD, further confirm- cement gradients is a core assumption of the dynamic ing its utility as an animal model of ADHD [13]. developmental theory of ADHD [38,75,76]. Regarding WIS rats, the intermediate length (in epoch 1) and fre- Alternative sources of hyperactivity quency (in both epochs) of their bouts suggest an inter- SHR produced shorter bouts at a higher rate than WKY mediate delay-of-reinforcement gradient for this strain over the range of VI schedules tested in the present relative to SHR and WKY. study. Prior research [42-45] suggests an interpretation The emission of short free-operant bouts, which sup- of these differences in terms of delay-of-reinforcement ports the delay-of-reinforcement-gradient hypothesis of gradients. Based on such interpretation, it would be operant hyperactivity, has recently been observed in expected that appending a tandem ratio requirement to the VI schedule of SHR would reduce the difference in its performance relative to WKY. But aside from length- ening bouts and reducing their frequency, a tandem requirement also increases within-bout responding, which has the net effect of increasing overall response rate [42,44,45]. That is, the tandem-ratio “treatment” is expected to increase SHR activity, not decrease it. This means that, although steeper delay-of-reinforcement gradients may be the main source of SHR overactivity, it is unlikely to be the only one. Two additional sources are possible: Increased motor capacity in SHR Estimates of Herrnstein’s hyperbola parameters ruled Figure 7 Delay-of-reinforcement-gradient hypothesis of SHR out motor capacity as a source of SHR hyperactivity, on hyperactivity. Ticks on the x-axis are responses; thick ticks are bout the basis of projected asymptotic response rates (k). It initiations. A reinforcer is delivered after the last response on the right of each panel. The sloped curves indicate that reinforcement is appears intuitive that such asymptotic rates reflect more effective with temporal proximity to the reinforcer. For SHR, motoric constraints in performance. Nonetheless, SHR only short bouts are effectively reinforced (panel A); for WKY, longer andWIS within-boutresponses (the faster response bouts are reinforced (panel B). Note that reinforcement affects more class) constituted only about half of the responses at the behaviors in panel B than in panel A, which entails that a smaller highest rate of reinforcement, when response rates were proportion of reinforcement strengthens bout initiation in panel B than in panel A. Therefore, relative to competing behaviors such as nearly asymptotic. This means that SHR and WIS could activities between bouts and responses within bouts, bout respond faster than what Herrnstein’s k suggests. Para- initiations are less effectively reinforced when reinforcement meter w is probably a more realistic reflection of moto- gradients are flatter. ric constraint. Estimates of the highest within-bout Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 12 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 response rate across schedules (Figure 6, middle panels) multiple-schedule context. Such close resemblance sug- suggest somewhat higher motoric capacity in young gests that differences between SHR and WKY perfor- SHR (PND 58-62), and limited motoric capacity in adult mance are robust against confounding factors in the WIS (PND 89-93). Although estimates of w are compro- present study. Second, the critical differences in bout- mised by the short bouts produced by SHR and WIS, and-pause parameters between strains (high p in WKY, elevated motoric capacity may not be ruled out as a high b is SHR) were not schedule-dependent. potential source of SHR hyperactivity, at least not dur- Differentiation between within-bout response rate and ing the transition from adolescence to adulthood. bout-initiation rate Furthermore, the notion that motoric capacity is Of the 90 rat × schedule estimations of bout-and-pause involved in SHR hyperactivity is consistent with prior parameters in each epoch, 46 in epoch 1 and 20 in data showing that IRTs shorter than 0.4 s are more fre- epoch 2 yielded p = 0 or 1. In those cases, the distribu- quently emitted by adult SHR than by adult WKY in VI tion of IRTs did not resemble a mixture of two expo- 30 s [34]. nentials (Equation 3) but just a single exponential. This SHR hypermotivation is noticeable in the nearly linear (in logarithmic scale) Although the interpretation of differences in bout-initia- IRT survivor plots shown in Figure 5, particularly those tion rate (b) in terms of incentive motivation is condi- of SHR and WIS and of rich schedules. Exponential IRT tional to the absence of changes in other parameters distributions yielded ambiguous estimates of p and inde- (see A delay-of-reinforcement-gradient hypothesis), terminate estimates of either w or b (footnote 1 clarifies changes in p and w do not rule out differences in incen- how it was chosen between p = 0 and p = 1 in each esti- tive motivation. In fact, it appears that the reduction in mation). Despite consistent differences in parameters b that is expected from a tandem ratio treatment would across schedules and strains, the uncertainty regarding be too small to reduce SHR estimates to WKY levels: parameter estimates implies that inferences drawn from Brackney and colleagues [45] report that a tandem ratio them, particularly in epoch 1, should be taken with cau- requirement reduced b by 37% in a VI 120 s. In a com- tion. Although the short length of SHR bouts is itself a parable schedule (VI 96 s), estimates of b in WKY were very important finding, future research should promote 71% shorter than SHR in epoch 1, and 78% shorter in longer bouts by imposing small tandem ratio require- epoch 2. Therefore, it seems likely that incentive moti- ments to all strains. This methodological adjustment vation differences in b by itself-also contributed to oper- would make pauses between and within bouts more ant hyperactivity in SHR. readily distinguishable. Confound of training experience and age Limitations The parameters of Herrnstein’s hyperbola and the bout- Finally, we acknowledge and address three potential lim- and-pause model were examined in two epochs, PND itations of the present study. These limitations do not 58-62 and 89-93. Between epochs, response rates compromise the basic conclusions inferred from the increased across schedules in SHR, only at high reinfor- data, but constrain the interpretation of the present cement rates in WKY, and not visibly in any schedule in results in terms of underlying psychological and devel- WIS. These divergent patterns of change over time exa- opmental processes. cerbated the differences in response rate between SHR Lack of stimulus control and control strains at low rates of reinforcement. The The first latency in each VI component (Latency 1 in elevated rate of weakly reinforced responses is the signa- Figure 4) did not vary systematically with rate of reinfor- ture of operant hyperactivity in SHR [32,33]. One possi- cement for any strain. This indicates that the tone asso- ble implication of the present results is that ciated with each VI schedule was not effective in hyperactivity emerges more strongly with adulthood. controlling response rate. Differences in response rate Although past research is consistent with these results across schedules depended on adjustments of response [34], they do not appear to be consistent with the modal rate to local rate of reinforcement. Such adjustments developmental trajectory of hyperactivity in ADHD may have introduced extraneous variability in VI perfor- [78-80]. Impulsive-hyperactive symptoms associated mance among strains. Furthermore, even if the tones with ADHD generally decline with age. Note, however, had been effective discriminative stimuli, schedule inter- that thepresent studydid notexamine ageseparately actions might have also confounded our results. These from training experience: older, more hyperactive SHR limitations, however, do not appear to seriously compro- had more exposure to the schedules of reinforcement mise the findings of the present study, for two reasons. than younger, less hyperactive SHR. The inconsistency First, the changes in overall response rate as a function between hyperactivity in SHR and in ADHD may stem of rate of reinforcement and strain resemble those from this confound. Our data, in fact, points at a possi- ble coincidence between the developmental trajectories observed before [32,33], which were not collected in a Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 13 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 of SHR and ADHD hyperactivity: SHR learned to (or solving for B, matured to, we do not know) produce longer response 1 1 L bouts, a pattern that was more typical of WKY controls. (A3) B = + − . T t Tt Although this study was not primarily aimed at discri- minating between practice and maturational effects on Based on the assumed distribution of IRTs (Equation operant hyperactivity, it provides, nonetheless, hints that 3), and assuming a minimum response duration δ (0.11 may guide future research on developmental factors s in this report), the mean IRT is involved in ADHD. p (1 − p) t = + + δ. (A4) Conclusions w b This study confirms that operant hyperactivity in SHR, Mean response rate in a VI schedule may thus be a purported animal model of ADHD, is expressed only recovered by substituting t in Equation A3 with the at low rates of reinforcement. This effect was observed right-hand side of Equation A4, and assuming that T in the transition from adolescence to adulthood (PND equals the VI requirement I. A more precise estimate of 58-62) and, more markedly, during early adulthood T is (PND 89-93). A close examination of the microstruc- ture of VI performance indicates that, across ages and T = I + . (A5) schedules, operant hyperactivity in SHR may be due to steeper delay-of-reinforcement gradients relative to control strain WKY. Inordinate motivation for incen- Endnotes tives and elevated motoric capacity may also contribute Theestimateof p for several rats under various VI to operant hyperactivity in SHR. With adulthood, schedules was 1.0 or zero, which posed a problem for delay-of-reinforcement gradients in SHR appear to flat- parameter estimation. Whether p =1.0 or zero,or w = ten; its motoric capacity becomes hardly distinguish- b, Equation 1 is reduced to an exponential density func- able from WKY, but its motivation for highly valued tion, with either p =1.0,and b not computable (i.e., incentives, such as sucrose pellets, grows even stron- bouts are infinitely long) or p =0,and w not computa- ger. Whether these changes in performance parameters ble (i.e., bouts are 1 lever press long). These two situa- are due to training experience, maturation, or a combi- tions are not distinguishable. When p =1.0 or p =0 nation of both, is yet unclear. These results suggest, had to be chosen for a particular rat, the variance with nonetheless, that complex and important learning, respect to p estimates in other VI schedules within the motivational, and developmental processes expressed same subject were taken into consideration. The esti- in SHR behavior appear to underlie operant hyperac- mate of the ambiguous p was the one that minimized tivity in ADHD. thevarianceamong p estimates. When p could take either value, 1.0 or zero, in all 5 VI schedules (this hap- Appendix: Computing mean response rate from pened in 7 of 36 rat × epoch observations), p was invari- its components ably estimated to be zero, because under such Response rate B over an interval T is the number of assumption the mean estimate of b for these rats (0.54 responses made in that interval (N) divided by T. There- resp/sec) was closer to the mean estimate of b for other fore, rats and epochs (0.42 resp/sec) than to the estimates of N = BT (A1) w for other rats and epochs (2.28 resp/sec). Brackney and colleagues (2011) considered δ, the shortest possible If T is the interval between trial onset and reinforce- IRT, a better estimation of motoric constraint than w. ment, then T may be partitioned into two periods: the Because of the low temporal resolution at which time between trial onset and the first response (latency, responses were recorded in the present study (9 Hz), δ or L)and thetimebetween thefirst response andthe could not be analyzed separately. It is likely, however, reinforced response (T - L). The latter may be further that variations in w between strains comprise variations portioned out into N - 1 inter-response times (IRTs). in δ, particularly at high values of w. The mean IRT is t =(T - L)/(N -1). Solvingfor T in the mean IRT equation and then substituting N with BT, List of Abbreviations ADHD: Attention deficit hyperactivity disorder; SHR: Spontaneously T = L +(BT − 1)t; (A2) hypertensive rat; WKY: Wistar Kyoto rat; WIS: Wistar rat; VI: Variable interval; IRT: Inter-response time; AICc: Corrected Akaike Information Criterion. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 14 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 19. van den Bergh FS, Bloemarts E, Chan JSW, Groenink L, Olivier B, Oosting RS: Acknowledgements and funding Spontaneously hypertensive rats do not predict symptoms of attention- The authors would like to thank Peter Killeen for helpful comments on deficit hyperactivity disorder. Pharmacol Biochem Behav 2006, 83:380-390. analyses and earlier drafts of the manuscript. We thank Jonathan Schiro and 20. Paulus M, Geyer M: Three independent factors characterize spontaneous Alison Moritz for help with figure editing. rat motor activity. Behav Brain Res 1993, 53:11-20. This project was supported by startup funds from the College of Liberal Arts 21. Ferster C, Skinner B: Schedules of Reinforcement. New York: Appleton- and Sciences, Arizona State University, to FS. Century-Crofts; 1957. 22. Hsieh Y, Yang C: Age-series characteristics of locomotor activities in Authors’ contributions spontaneously hypertensive rats: a comparison with the Wistar-Kyoto FS conceived and designed the study. JH and KH acquired and analyzed strain. Physiol Behav 2008, 93:777-782. data. JH, KH and FS interpreted data and drafted and revised the 23. Knardah S, Sagvolden T: Open-field behavior of spontaneously- manuscript. Parts of the manuscript served as Honors thesis for KH. All authors read and approved the final manuscript. hypertensive rats. Behav Neural Biol 1979, 27:187-200. 24. Gentsch C, Lichtsteiner M, Feer H: Open field and elevated plus-maze: A Competing interests behavioral comparison between spontaneously hypertensive (SHR) and The authors declare that they have no competing interests. Wistar-Kyoto (WKY) rats and the effects of chlordiazepoxide. Behav Brain Res 1987, 25:101-107. Received: 26 September 2011 Accepted: 26 January 2012 25. Sagvolden T, Hendley ED, Knardahl S: Behavior of hypertensive and Published: 26 January 2012 hyperactive rat strains: hyperactivity is not unitarily determined. Physiol Behav 1992, 52:49-57. 26. Myers M, Musty R, Hendley E: Attenuation of hyperactivity in the References spontaneously hypertensive rat by amphetamine. Behav Neural Biol 1982, 1. Froehlich T, Lanphear BP, Epstein JN, Barbaresi WJ, Katusic SK, Kahn RS: 34:42-54. Prevalence, recognition, and treatment of attention-deficit/hyperactivity 27. Danysz W, Plaźnik A, Pucilowski O, Plewako M, Obersztyn M, Kostowski W: disorder in a national sample of US children. Arch Pediatr Adolesc Med Behavioral studies in spontaneously hypertensive rats. Behav Neural Biol 2007, 161(9):857-864. 1983, 39:22-29. 2. Polanczyk G, de Lima, Horta B, Biederman J, Rohde L: The worldwide 28. Cierpial M, M McCarty R: Maternal effects on the development of prevalence of ADHD: a systematic review and metaregression analysis. spontaneous hypertension. Health Psychol 1988, 7:125-135. Am J Psychiatry 2007, 164:942-948. 29. Ferguson S, Cada A: A longitudinal study of short- and long-term activity 3. Kessler RC, Adler L, Barkley R, Biederman J, Conners K, Demler O, levels in male and female spontaneously hypertensive, Wistar-Kyoto, Faraone SV, Greenhill LL, Howes MJ, Secnik K, Spencer T, Ustun TB, and Sprague-Dawley rats. Behav Neurosci 2003, 117:271-282. Walters EE, Zaslavsky AM: The prevalence and correlates of adult ADHD in Amini B, Swann A, Dafny N: Differential locomotor responses in male rats the United States: results from the national comorbidity survey from three strains to acute methylphenidate. Int J Neurosci 2004, replication. Am J Psychiatry 2006, 163:716-723. 114:1063-1084. 4. Faraone SV, Sergeant J, Gillberg C, Biederman J: The worldwide prevalence 31. Sagvolden T, Pettersen M, Larsen M: Spontaneously hypertensive rates of ADHD: is it an American condition? World Psychiatry 2003, 2(2):104-113. (SHR) as a putative animal model of childhood hyperkinesis: SHR 5. American Psychiatric Association: Diagnostic and statistical manual of behavior compared to four other rat strains. Physiol Behav 1993, mental disorders. Washington, DC: Author;, 4 2000. 54(6):1047-1055. 6. Barkley RA: Behavioral inhibition, sustained attention, and executive 32. Sagvolden T, Metzger MA, Sagvolden G: Frequent rewards eliminates functions: constructing a unifying theory of ADHD. Psychol Bull 1997, differences in activity between hyperkinetic rats and controls. Behav 121:65-94. Neurol Biol 1993, 59:225-229. 7. Barry TD, Lyman RD, Klinger LG: Academic under-achievement and 33. Williams J, Sagvolden G, Taylor E, Sagvolden T: Dynamic behavioral attention-deficit/hyperactivity disorder: the negative impact of symptom changes in the Spontaneously Hypertensive Rat 1. Control by place, severity on school performance. J School Pscyhol 2002, 40(3):259-283. timing, and reinforcement rate. Behav Brain Res 2009, 198:273-282. 8. Spira EG, Fischel JE: The impact of preschool inattention, hyperactivity, 34. Johansen E, Sagvolden S, Kvande G: Effects of delayed reinforcers on the and impulsivity on social and academic development: a review. J Child behavior of an animal model of attention-deficit/hyperactivity disorder Psychol Psych 2005, 46:755-773. (ADHD). Behav Brain Res 2005, 162(1):47-61. 9. Loe IM, Feldmen HM: Academic and educational outcomes of children 35. Sagvolden T, Aase H, Zeiner P, Berger DF: Altered reinforcement with ADHD. Ambul Pediatr 2007, 7(1):82-90. mechanisms in ADHD. Behav Brain Res 1998, 33:387-398. 10. Sagvolden T, Russell VA, Aase H, Johansen EB, Farshbaf M: Rodent models 36. Douglas V, Parry P: Effects of reward on delayed reaction time task of attention-deficit/hyperactivity disorder. Biol Psychiatry 2005, performance in hyperactive children. J Abnorm Child Psych 1983, 57:1239-1247. 11:313-326. 11. Russell V: Neurobiology of animal models of attention-deficit 37. Aase H, Sagvolden T: Infrequent, but not frequent, reinforcers produce hyperactivity disorder. J Neurosci Methods 2007, 161:185-198. more variable responding and deficient sustained attention in young 12. Bari A, Robbins TW: Animal models of ADHD. Molecular and Functional children with attention-deficit/hyperactivity disorder (ADHD). J Child Models in Neuropsychiatry: Curr Top Behav Neurosci 2011, 7:149-185. Psychol Psych 2006, 47:457-471. 13. Sanabria F, Killeen PR: Evidence for impulsivity in the Spontaneously 38. Sagvolden T, Johansen EB, Aase H, Russell VA: A dynamic developmental Hypertensive Rat drawn from complementary response-withholding theory of attention-deficit/hyperactivity disorder (ADHD) predominantly tasks. Behav Brain Funct 2008, 4:7-24. hyperactive/impulsive and combined subtypes. Behav Brain Sci 2005, 14. Fox AT, Hand DJ, Reilly MP: Impulsive choice in a rodent model of 28:397-419. attention-deficit/hyperactivity disorder. Behav Brain Res 2008, 187:146-152. 39. de Villiers P, Herrnstein RJ: Toward a law of response strength. Psych Bull 15. Evenden J, Meyerson B: The behavior of Spontaneously Hypertensive and 1976, 83:1131-1153. Wistar Kyoto rats under a paced fixed consecutive number schedule of 40. Herrnstein RJ: On the law of effect. J Exp Anal Behav 1970, 13(2):243-266. reinforcement. Pharmacol Biochem Behav 1999, 63:71-82. 41. Blough DS: Interresponse time as a function of continuous variables: a 16. Berger DF, Sagvolden T: Sex differences in visual discrimination new method and some data. J Exp Anal Behav 1963, 6:237-246. behaviour in an animal model of attention deficit hyperactivity disorder. 42. Shull RL, Gaynor ST, Grimes JA: Response rate viewed as engagement Behav Brain Res 1998, 94:73-82. bouts: effects of relative reinforcement and schedule type. J Exp Anal 17. Sagvolden T: Behavioral validation of the spontaneously hypertensive rat Behav 2001, 75:247-274. (SHR) as an animal model of attention-deficit/hyperactivity disorder (AD/ 43. Shull RL, Grimes JA, Bennett JA: Bouts of responding: the relation HD). Neurosci Biobehav Rev 2000, 24:31-39. between bout rate and the rate of variable-interval reinforcement. J Exp 18. Alsop B: Problems with spontaneously hypertensive rats (SHR) as a Anal Behav 2004, 81:65-83. model of attention-deficit/hyperactivity disorder (AD/HD). J Neurosci Methods 2007, 162:42-48. Hill et al. Behavioral and Brain Functions 2012, 8:5 Page 15 of 15 http://www.behavioralandbrainfunctions.com/content/8/1/5 44. Shull RL, Grimes JA: Bouts of responding from variable-interval 71. Johansen EB, Killeen PR, Sagvolden T: Behavioral variability, elimination of reinforcement of lever pressing by rats. J Exp Anal Behav 2003, responses, and delay-of-reinforcement gradients in SHR and WKY rats. 80:159-171. Behav Brain Funct 2007, 3:60. 45. Brackney RJ, Cheung TH, Neisewander JL, Sanabria F: The isolation of 72. Johansen EB, Sagvolden T: Slower extinction of responses maintained by motivational, motoric, and schedule effects on operant performance: a intra-cranial self-stimulation (ICSS) in an animal model of attention- modeling approach. J Exp Anal Behav 2011, 37:1-9. deficit/hyperactivity disorder (ADHD). Behav Brain Res 2005, 162:22-31. 46. Reed P: An experimental analysis of steady-state response rate 73. Sutherland KR, Alsop B, McNaughton N, Hyland BI, Tripp G, Wickens JR: components on variable ratio and variable interval schedules of Sensitivity to delay of reinforcement in two animal models of attention reinforcement. J Exp Psychol [Anim Behav] 2011, 39:27-35. deficit hyperactivity disorder (ADHD). Behav Brain Res 2009, 205:372-376. 47. Shull RL: Bouts, changeovers, and units of operant behavior. Eur J Behav 74. Sagvolden T, Aase H, Zeiner P, Berger D: Altered reinforcement Anal 2011, 12:49-72. mechanisms in attention-deficit/hyperactivity disorder. Behav Brain Res 48. Conover KL, Fulton S, Shizgal P: Operant tempo varies with reinforcement 1998, 94:61-71. rate: implications for measurement of reward efficacy. Behav Process 75. Johansen EB, Killeen PR, Russell VA, Tripp G, Wickens JR, Tannock R, 2001, 56:85-101. Williams J, Sagvolden T: Origins of altered reinforcement effects in ADHD. 49. Sagvolden T, Johansen E, Woien G, Walaas SI, Storm-Mathisen J, Behav Brain Funct 2009, 5:7. Bergerson LH, Hvalby O, Jensen V, Aase H, Russell V, Killeen PR, 76. Luman M, Tripp G, Scheres A: Identifying the neurobiology of altered DasBanerjee T, Middleton F, Faraone SV: The spontaneously hypertensive reinforcement sensitivity in ADHD: a review and research agenda. rat model of ADHD- The importance of selecting the appropriate Neurosci Biobehav Rev 2010, 34:744-754. reference strain. Neuropharmacology 2009, 57:619-626. 77. Taylor D, Lincoln AJ, Foster SL: Impaired behavior regulation under 50. Fleshler M, Hoffman HS: A progression for generating variable-interval conditions of concurrent variable schedules of reinforcement in children schedules. J Exp Anal Behav 1962, 5:529-530. with ADHD. J Attention Disord 2010, 13:358-368. 51. Spear LP, Brake SC: Periadolescence: Age-dependent behavior and 78. Hart EL, Lahey BB, Loeber R, Applegate B, Green SM, Frick PJ: pyschopharmacological responsivity in rats. Dev Psychobiol 2009, Developmental change in attention-deficit hyperactivity disorder in 16:83-111. boys: a four-year longitudinal study. J Abnorm Child Psych 1995, 52. Herrnstein RJ: Relative and absolute strength of responses as a function 23:729-749. of frequency of reinforcement. J Exp Anal Behav 1961, 4:267-272. 79. Faraone SV, Biederman J: What is the prevalence of adult ADHD? Results 53. Shull RL: The sensitivity of response rate to the rate of variable-interval of a population screen of 966 adults. J Attention Disord 2005, 9:384-391. reinforcement for pigeons and rats: a review. J Exp Anal Behav 2005, 80. Faraone S, Biederman J, Mick E: The age-dependent decline of attention- 84:99-110. deficit/hyperactivity disorder: a meta-analysis of follow-up studies. 54. Fantino E, Davison M: Choice: some quantitative relations. J Exp Anal Psychol Med 2006, 36:159-165. Behav 1983, 40(1):1-13. doi:10.1186/1744-9081-8-5 55. Bradshaw CM, Ruddle HV, Szabadi E: Relationship between response rate Cite this article as: Hill et al.: Characterizing operant hyperactivity in the reinforcement frequency in variable-interval schedules: III. the effect of Spontaneously Hypertensive Rat. Behavioral and Brain Functions 2012 8:5. d-amphetamine. J Exp Anal Behav 1981, 36(1):29-39. 56. Martens BK, Lochner DG, Kelly SQ: The effects of variable interval reinforcement on academic engagement: a demonstration of matching theory. J Exp Anal Behav 1992, 25(1):143-151. 57. Dallery J, Soto PL: Herrnstein’s hyperbolic matching equation and behavioral pharmacology: review and critique. Behav Pharmacol 2004, 15:443-459. 58. Burnham KP, Anderson DR: Model selection and multimodel inference: A practical information-theoretic approach New York NY, Springer-Verlag; 2002. 59. Myung I: Tutorial on maximum likelihood estimation. J Math Psychol 2003, 47:90-100. 60. Bowers MT, Hill J, Palya WL: Interresponse time structures in variable- ration and variable-interval schedules. J Exp Anal Behav 2008, 90:345-362. 61. Johnson JE, Pese EF, Newland MC: High-rate operant behavior in two mouse strains: a response-bout analysis. Behav Process 2009, 81:309-315. 62. Podlesnik CA, Jimenez-Gomez C, Ward RD, Shahan TA: Resistance to change of responding maintained by unsignaled delays to reinforcement: a response-bout analysis. J Exp Anal Behav 2006, 85:329-347. 63. Malott RW: The effects of prefeeding in plain and chained fixed ratio schedules of reinforcement. Psychon Sci 1966, 4:285-286. 64. Sidman M, Stebbins WC: Satiation effects under fixed-ratio schedules of reinforcement. J Comp Physiol Psychol 1954, 47:114-116. 65. Killeen P: Reinforcement frequency and contingency as factors in fixed- ratio behavior. J Exp Anal Behav 1969, 12(3):391-395. 66. Dickinson A, Watt A, Griffiths WJH: Free-operant acquisition with delayed Submit your next manuscript to BioMed Central reinforcement. Q J Exp Psychol-B 1992, 45:241-258. and take full advantage of: 67. Killeen PR, Fantino E: Unification of models for choice between delayed reinforcers. J Exp Anal Behav 1990, 53(1):189-200. • Convenient online submission 68. Killeen PR: Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid. Behav • Thorough peer review Process 2011, 87:57-63. • No space constraints or color figure charges 69. Lattal KA: Delayed reinforcement of operant behavior. J Exp Anal Behav • Immediate publication on acceptance 2010, 93:129-139. 70. Hand DJ, Fox AT, Reilly MP: Response acquisition with delayed • Inclusion in PubMed, CAS, Scopus and Google Scholar reinforcement in a rodent model of attention-deficit/hyperactivity • Research which is freely available for redistribution disorder (ADHD). Behav Brain Res 2006, 175:337-342. Submit your manuscript at www.biomedcentral.com/submit

Journal

Behavioral and Brain Functions – Springer Journals

Published: Jan 26, 2012

There are no references for this article.