Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Adaptive Experimental Design: Prospects and Applications in Political Science

Adaptive Experimental Design: Prospects and Applications in Political Science Experimental researchers in political science frequently face the problem of inferring which of several treatment arms is most effective. They may also seek to estimate mean outcomes under that arm, construct confidence intervals, and test hypotheses. Ordinarily, multiarm trials conducted using static designs assign participants to each arm with fixed probabilities. However, a growing statistical literature suggests that adaptive experimental designs that dynamically allocate larger assignment probabilities to more promising treatments are better equipped to discover the best performing arm. Using simulations and empirical applications, we explore the conditions under which such designs hasten the discovery of superior treatments and improve the precision with which their effects are estimated. Recognizing that many scholars seek to assess performance relative to a control condition, we also develop and implement a novel adaptive algorithm that seeks to maximize the precision with which the largest treatment effect is estimated . http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png American Journal of Political Science Wiley

Adaptive Experimental Design: Prospects and Applications in Political Science

Download PDF
19 pages

Loading next page...
 
/lp/wiley/adaptive-experimental-design-prospects-and-applications-in-political-T3W3Rk1mUv
Publisher
Wiley
Copyright
© 2021 by the Midwest Political Science Association
ISSN
0092-5853
eISSN
1540-5907
DOI
10.1111/ajps.12597
Publisher site
See Article on Publisher Site

Abstract

Experimental researchers in political science frequently face the problem of inferring which of several treatment arms is most effective. They may also seek to estimate mean outcomes under that arm, construct confidence intervals, and test hypotheses. Ordinarily, multiarm trials conducted using static designs assign participants to each arm with fixed probabilities. However, a growing statistical literature suggests that adaptive experimental designs that dynamically allocate larger assignment probabilities to more promising treatments are better equipped to discover the best performing arm. Using simulations and empirical applications, we explore the conditions under which such designs hasten the discovery of superior treatments and improve the precision with which their effects are estimated. Recognizing that many scholars seek to assess performance relative to a control condition, we also develop and implement a novel adaptive algorithm that seeks to maximize the precision with which the largest treatment effect is estimated .

Journal

American Journal of Political Science – Wiley

Published: Oct 1, 2021

References

  • Beyond the Running Tally: Partisan Bias in Political Perceptions
    Bartels, Larry
  • Bandit Problems: Sequential Allocation of Experiments
    Berry, Donald A.; Fristedt, Bert
  • Unbiased Estimation for Response Adaptive Clinical Trials
    Bowden, Jack; Trippa, Lorenzo
  • Partisan Bias in Factual Beliefs about Politics
    Bullock, John G.; Gerber, Alan S.; Hill, Seth J.; Huber, Gregory A.
  • The American Voter
    Campbell, Angus; Converse, Philip E.; Miller, Warren E.; Stokes, Donald E.
  • Adaptive and Flexible Clinical Trials
    Chin, Richard
  • Adaptive Design Methods in Clinical Trials—A Review
    Chow, Shein‐Chung; Chang, Mark
  • Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
    Coppock, Alexander
  • Avoiding Post‐Treatment Bias in Audit Experiments
    Coppock, Alexander
  • Validating the Demographic, Political, Psychological, and Experimental Results Obtained from a New Source of Online Survey Respondents
    Coppock, Alexander; McClellan, Oliver A.
  • Generalizability of Heterogeneous Treatment Effect Estimates Across Samples
    Coppock, Alexander; Leeper, Thomas J.; Mullinix, Kevin J.
  • Field Experiments: Design, Analysis, and Interpretation
    Gerber, Alan S.; Green, Donald P.
  • Partisanship, Political Control, and Economic Assessments
    Gerber, Alan S.; Huber, Gregory A.
  • Proceedings of the 27th International Conference on Machine Learning (ICML 2010) (ICML‐10)
    Graepel, Thore; Candela, Joaquin Quiñonero; Borchert, Thomas; Herbrich, Ralf
  • 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2010, Cordoba, Spain, June 1‐4, 2010, Proceedings, Part III (IEA‐AIE 2010)
    Granmo, Ole‐Christoffer; Berg, Stian
  • Proceedings of the 2011 10th International Conference on Machine Learning and Applications and Workshops ‐ Volume 01
    Gupta, Neha; Granmo, Ole‐Christoffer; Agrawala, Ashok
  • Who Gets a Swiss Passport? A Natural Experiment in Immigrant Discrimination
    Hainmueller, Jens; Hangartner, Dominik
  • Motivated Responding in Studies of Factual Learning
    Khanna, Kabir; Sood, Gaurav
  • Bandit: Functions for Simple A/B Split Test and Multi‐Armed Bandit Analysis
    Lotze, Thomas; Loecher, Markus
  • Mechanism Experiments and Policy Evaluations
    Ludwig, Jens; Kling, Jeffrey R.; Mullainathan, Sendhil
  • How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It
    Montgomery, Jacob M.; Nyhan, Brendan; Torres, Michelle
  • The Generalizability of Survey Experiments
    Mullinix, Kevin J.; Leeper, Thomas J.; Druckman, James N.; Freese, Jeremy
  • Optimal Dynamic Treatment Regimes
    Murphy, Susan A
  • Monitoring Corruption: Evidence from a Field Experiment in Indonesia
    Olken, Benjamin A
  • Adaptive Designs in Clinical Trials: Why UseThem, and How to Run and Report Them
    Pallmann, Philip; Bedding, Alun W.; Choodari‐Oskooei, Babak; Dimairo, Munyaradzi; Flight, Laura; Hampson, Lisa V.; Holmes, Jane; Mander, Adrian P.; Sydes, Matthew R.; Villar, Sofía S.
  • Money, Time, and Political Knowledge: Distinguishing Quick Recall and Political Learning Skills
    Prior, Markus; Lupia, Arthur
  • You Cannot Be Serious: The Impact of Accuracy Incentives on Partisan Bias in Reports of Economic Perceptions
    Prior, Markus; Sood, Gaurav; Khanna, Kabir
  • Reinforcement Learning: An Introduction
    Sutton, Richard S.; Barto, Andrew G.
  • Flexible Trial Design in Practice‐Stopping Arms for Lack‐of‐Benefit and Adding Research Arms Mid‐Trial in STAMPEDE: A Multi‐Arm Multi‐Stage Randomized Controlled Trial
    Sydes, Matthew R.; Parmar, Mahesh K. B.; Mason, Malcolm D.; Clarke, Noel W.; Amos, Claire; Anderson, John; Bono, Johann; Dearnaley, David P.; Dwyer, John; Green, Charlene; Jovic, Jordana; Ritchie, Alistair W. S.; Martin Russell, J.; Sanders, Karen; Thalmann, George; James, Nicholas D.
  • On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Aamples
    Thompson, William R
  • On the Theory of Apportionment
    Thompson, William R
  • Bayesian Adaptive Randomized Trial Design for Patients with Recurrent Glioblastoma
    Trippa, Lorenzo; Lee, Eudocia Q.; Wen, Patrick Y.; Batchelor, Tracy T.; Cloughesy, Timothy; Parmigiani, Giovanni; Alexander, Brian M.
  • Multi‐Armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges
    Villar, Sofía S.; Bowden, Jack; Wason, James
  • A Comparison of Bayesian Adaptive Randomization and Multi‐Stage Designs for Multi‐Arm Clinical Trials
    Wason, James M.S.; Trippa, Lorenzo
  • When to Keep It Simple—Adaptive Designs Are Not Always Useful
    Wason, James M.S.; Brocklehurst, Peter; Yap, Christina
  • Exact Two‐Sample Permutation Tests Based on the Randomized Play‐the‐Winner Rule
    Wei, L.J.
  • Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results
    Young, Alwyn