Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Advances in Longitudinal HCI ResearchExperiments, Longitudinal Studies, and Sequential Experimentation: How Using “Intermediate” Results Can Help Design Experiments

Advances in Longitudinal HCI Research: Experiments, Longitudinal Studies, and Sequential... [This chapter formalizes the traditional randomized experiment as a sequential decision problem in which treatments are allocated to units sequentially to achieve a specific goal. This problem description is known as the multi-armed bandit (MAB) problem and we describe it in detail and relate it to the methodological considerations that arise when designing longitudinal studies in HCI. Subsequently, the chapter reviews multiple treatment allocation policies—attempts to solve the MAB problem—and analyzes their properties. Next, we discuss utility of a sequential perspective on experimentation for various methodological purposes such as early stopping, best arm selection, and powerful testing. We demonstrate how in many cases, and particularly in longitudinal studies, the “intermediate” results of an experiment can be used to improve the experimental design. We close off by discussing several recent software packages that allow readers to implement and analyze sequential experiments.] http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png

Advances in Longitudinal HCI ResearchExperiments, Longitudinal Studies, and Sequential Experimentation: How Using “Intermediate” Results Can Help Design Experiments

Part of the Human–Computer Interaction Series Book Series
Editors: Karapanos, Evangelos; Gerken, Jens; Kjeldskov, Jesper; Skov, Mikael B.
Kaptein, Maurits
Advances in Longitudinal HCI Research — Aug 12, 2021
Download PDF
28 pages

Loading next page...
 
/lp/springer-journals/advances-in-longitudinal-hci-research-experiments-longitudinal-studies-tq0yPagEW5
Publisher
Springer International Publishing
Copyright
© Springer Nature Switzerland AG 2021
ISBN
978-3-030-67321-5
Pages
121 –149
DOI
10.1007/978-3-030-67322-2_7
Publisher site
See Chapter on Publisher Site

Abstract

[This chapter formalizes the traditional randomized experiment as a sequential decision problem in which treatments are allocated to units sequentially to achieve a specific goal. This problem description is known as the multi-armed bandit (MAB) problem and we describe it in detail and relate it to the methodological considerations that arise when designing longitudinal studies in HCI. Subsequently, the chapter reviews multiple treatment allocation policies—attempts to solve the MAB problem—and analyzes their properties. Next, we discuss utility of a sequential perspective on experimentation for various methodological purposes such as early stopping, best arm selection, and powerful testing. We demonstrate how in many cases, and particularly in longitudinal studies, the “intermediate” results of an experiment can be used to improve the experimental design. We close off by discussing several recent software packages that allow readers to implement and analyze sequential experiments.]

Published: Aug 12, 2021

Keywords: Sequential experimentation; Multi-armed bandits; Thompson sampling; StreamingBandit; Contextual

There are no references for this article.