Get 20M+ Full-Text Papers For Less Than $1.50/day. Subscribe now for You or Your Team.

Learn More →

Adjusting Treatment Effect Estimates by Post-Stratification in Randomized Experiments

Adjusting Treatment Effect Estimates by Post-Stratification in Randomized Experiments SummaryExperimenters often use post-stratification to adjust estimates. Post-stratification is akin to blocking, except that the number of treated units in each stratum is a random variable because stratification occurs after treatment assignment. We analyse both post-stratification and blocking under the Neyman–Rubin model and compare the efficiency of these designs. We derive the variances for a post-stratified estimator and a simple difference-in-means estimator under different randomization schemes. Post-stratification is nearly as efficient as blocking: the difference in their variances is of the order of 1/n2, with a constant depending on treatment proportion. Post-stratification is therefore a reasonable alternative to blocking when blocking is not feasible. However, in finite samples, post-stratification can increase variance if the number of strata is large and the strata are poorly chosen. To examine why the estimators’ variances are different, we extend our results by conditioning on the observed number of treated units in each stratum. Conditioning also provides more accurate variance estimates because it takes into account how close (or far) a realized random sample is from a comparable blocked experiment. We then show that the practical substance of our results remains under an infinite population sampling model. Finally, we provide an analysis of an actual experiment to illustrate our analytical results. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of the Royal Statistical Society Series B (Statistical Methodology) Oxford University Press

Adjusting Treatment Effect Estimates by Post-Stratification in Randomized Experiments

Loading next page...
 
/lp/oxford-university-press/adjusting-treatment-effect-estimates-by-post-stratification-in-bibjQdplbJ

References (53)

Copyright
© 2012 Royal Statistical Society
ISSN
1369-7412
eISSN
1467-9868
DOI
10.1111/j.1467-9868.2012.01048.x
Publisher site
See Article on Publisher Site

Abstract

SummaryExperimenters often use post-stratification to adjust estimates. Post-stratification is akin to blocking, except that the number of treated units in each stratum is a random variable because stratification occurs after treatment assignment. We analyse both post-stratification and blocking under the Neyman–Rubin model and compare the efficiency of these designs. We derive the variances for a post-stratified estimator and a simple difference-in-means estimator under different randomization schemes. Post-stratification is nearly as efficient as blocking: the difference in their variances is of the order of 1/n2, with a constant depending on treatment proportion. Post-stratification is therefore a reasonable alternative to blocking when blocking is not feasible. However, in finite samples, post-stratification can increase variance if the number of strata is large and the strata are poorly chosen. To examine why the estimators’ variances are different, we extend our results by conditioning on the observed number of treated units in each stratum. Conditioning also provides more accurate variance estimates because it takes into account how close (or far) a realized random sample is from a comparable blocked experiment. We then show that the practical substance of our results remains under an infinite population sampling model. Finally, we provide an analysis of an actual experiment to illustrate our analytical results.

Journal

Journal of the Royal Statistical Society Series B (Statistical Methodology)Oxford University Press

Published: Dec 4, 2012

Keywords: Blocking; Neyman; Rubin model; Randomized trials; Regression adjustment

There are no references for this article.