Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Can Consumers Make Affordable Care Affordable? The Value of Choice Architecture

Can Consumers Make Affordable Care Affordable? The Value of Choice Architecture Tens of millions of people are currently choosing health coverage on a state or federal health insurance exchange as part of the Patient Protection and Affordable Care Act. We examine how well people make these choices, how well they think they do, and what can be done to improve these choices. We conducted 6 experiments asking people to choose the most cost- effective policy using websites modeled on current exchanges. Our results suggest there is significant room for improvement. Without interventions, respondents perform at near chance levels and show a significant bias, overweighting out-of-pocket expenses and deductibles. Financial incentives do not improve performance, and decision-makers do not realize that they are performing poorly. However, performance can be improved quite markedly by providing calculation aids, and by choosing a ‘‘smart’’ default. Implementing these psychologically based principles could save purchasers of policies and taxpayers approximately 10 billion dollars every year. Citation: Johnson EJ, Hassin R, Baker T, Bajger AT, Treuer G (2013) Can Consumers Make Affordable Care Affordable? The Value of Choice Architecture. PLoS ONE 8(12): e81521. doi:10.1371/journal.pone.0081521 Editor: Thomas Boraud, Centre national de la recherche scientifique, France Received March 28, 2013; Accepted October 14, 2013; Published December 18, 2013 Copyright:  2013 Johnson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This work was supported by the Alfred P. Sloan and Russell Sage Foundations (Grant number: 2011-5-12-ECON). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. * E-mail: ejj3@columbia.edu support decision-making. In this paper, we examine three related Introduction questions: Can people select the best policies? Do they know how Currently tens of millions of Americans, along with members of well they are doing? Does the design of the sites change their Congress and their staffs, are participating in a grand experiment performance? in consumer choice: They will select health insurance using a Our results suggest there is significant room to improve these marketplace or health insurance exchange operated by states and decisions. Without any intervention, respondents perform at near federal governments as part of the Patient Protection and chance levels and show a significant bias, overweighting out-of- Affordable Care Act. The success of these exchanges depends pocket costs and deductibles. Financial incentives do not improve upon two related premises: First that consumers will be able to performance, and decision-makers do not realize that they are select the best policy for their needs, and second that price performing badly. Without aids, only one population examined competition, driven by effective consumer choice, will lower here, Columbia MBA students, perform reasonably well at this prices. This hope is shared by divergent participants: Kathleen task. However, performance can be markedly improved by Sibelius, Secretary of Health and Human Services, and a providing calculation aids, and by choosing a ‘‘smart’’ default, Democrat, characterizes an exchange as ‘‘… a transparent, level raising the performance of ordinary respondents to that of the playing field, driving down costs; … giv[ing] individuals and small MBA students. businesses the same purchasing power as big businesses and a choice of plans to fit their needs.’’ [1] Bill Frist, a physician and Prior Research former Republican Senate Majority Leader, argues ‘‘State The quality of choices on prior health insurance exchanges has exchanges are good from a conservative standpoint because they been, at best, mixed. For example, when examining the exchanges involve consumer choice and markets.’’ [2]. implementing Medicare Part D, a prescription drug plan for These premises are critical not only to the new exchanges, but seniors, Heiss, McFadden, and Winter [3] conclude, ‘‘consumers also for all government administered health insurance markets and are likely to have difficulty choosing among plans to fine tune their for the efficiency of privately provided benefit choices – both prescription drug coverage.’’ Abaluck and Gruber [4] find that health exchanges and employer sponsored insurance center on only 12.2% of seniors pick the most cost-effective plan. consumer choice in finding plans that are cost-effective and While the economic analysis of choice suggests that issues appropriate for consumer needs and both include many design surrounding incentives and information may determine success, a decisions that will affect choice. Yet a large literature in psychology more psychological analysis suggests that good decisions depend, suggests that these premises may not be realistic, since, as we shall critically, on subtle elements of how the choices are presented to see, these exchanges may not provide a helpful choice architecture to PLOS ONE | www.plosone.org 1 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges the consumer, as described in an evolving literature on choice see Methods S1 for a thorough explanation of the experimental procedure. architecture [5–7]. Designing an exchange involves many design decisions including specifying the number and kind of options and Even in this simplified version of health insurance choice, the attributes offered, determining the arrangement of options and the process of selecting the most cost-effective health insurance is not format and order of attributes, and selecting default options and trivial, consumers must, for each plan: computational aids. 1. Consider the total premiums for the year, The Massachusetts ‘‘Connector,’’ an exchange operating since 2. Combine the copayments and the expected number of visits, 2006, illustrates the impact of choice architecture: Before late 2009, the Connector simultaneously presented 25 plans from 6 and insurance providers. In 2009, plans were reorganized into 3 tiers of 3. Include the minimum of the deductible and their out-of-pocket coverage, categorized by premiums and out-of-pocket costs. costs. Consumers first chose one of these levels and then viewed a For equal monthly premiums this is (12*Monthly Premium) + smaller set of 6 standardized plans within a level. Work by Ericson and Starc [8] shows that this simple change markedly altered (N of visits * Copay) + min(Out-of-Pocket Costs, Deductible). Adding risk considerations, while undoubtedly important, would behavior: Consumers were increasingly sensitive to premium costs and out-of-pocket costs, changing market shares for some carriers make these calculations even more difficult, thus making performance worse, not better than we observe, and perhaps by a factor of 2. make our interventions more effective. Thus, the advent of health exchanges presents a challenge: The The reader might consider selecting the most cost effective plan choice could be daunting for consumers, resulting in suboptimal in Figure 1, assuming, as did respondents in one of our choices of policies that provide the wrong features or are too experimental conditions, that he or she will make 9 doctor visits expensive. We are interested in how a prudent design of health exchanges based on psychological research could improve choice. and incur $900 in out-of-pocket costs in the upcoming year. This calculation might seem difficult, but some would argue that there We are also interested in a parallel question: Do people know if they are making good decisions? This is important because if might be heuristic strategies that perform well [9]. Yet we feel that there are two reasons for concern: First, users of these exchanges people know that they are not doing well, they could seek assistance, potentially remedying their poor performance. If will be largely unfamiliar with selecting health insurance – since many, 97% according to some estimates [7], will be buying health people are unaware of their inadequate performance, simply insurance for the first time and may lack experience and relevant providing access to assistance will not improve their decision- knowledge – and will not be highly educated (seventy-seven making. percent will have a High School diploma or less) [10]. Second, this is an economically significant decision for these households: Even Methods: Choosing health insurance with subsidies, premiums will represent between 4 and 9.5% of the When choosing insurance, consumers face two tasks. The first, modest median income of $48,529 for a family of 4 [10]. which we do not examine, is to estimate their expected usage and Consequently, mistakes may have large economic consequences. out-of-pocket expenses for the upcoming year, and to consider the uncertainty around these quantities as a choice under risk. The Results and Discussion: Can Consumers Choose second is to select the right plan given their expected usage. The Right Plan? Our studies focused on people’s ability to select cost-effective policies and remove risk and usage prediction considerations. We examined consumers’ decision-making abilities and condi- While economists analyze insurance choice by examining uncer- tions that might facilitate better decisions in a series of six framed tainty, risk, and asymmetric information, we investigated the field experiments [11], all but one using participants with impact of psychological variables such as calculation costs as a demographics similar to those projected to use the exchanges. In major barrier to better choices. We examined a simplified version addition to specifying the number of doctor visits one would make of the health insurance choice that allowed us to assess the and the out-of-pocket costs one would incur in a given year, we performance of choice architecture interventions, much like a also limited the number of plans available to either 4 or 8, a figure wind tunnel might be used to evaluate candidate airplane designs – markedly lower than the number to be used in future exchanges Figure 1. A decision display used in Experiment 4. Respondents saw either 8 (pictured) or 4 options. doi:10.1371/journal.pone.0081521.g001 PLOS ONE | www.plosone.org 2 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges Figure 2. The percentage of choices of the most cost effective option and respondents’ average error. The top half of each bar, in blue, represents the proportion of correct choices, and the bottom half, below the zero line in red, plots the average dollar error, across respondents. A dashed line for each condition represents the performance of a random chooser, and the error bars represent 95% confidence intervals. Darker shades denote the provision of calculators. Panel (A) represents the results of Experiments 1–4 collapsing across other manipulations (see SM). Panel (B) represents the results of a sample of highly educated MBA students (Experiment 5), and of individuals from the target population, when given different choice architecture interventions. For (b) the random response threshold ($1264) exceeds the lower limit of the graph. doi:10.1371/journal.pone.0081521.g002 (e.g., the Massachusetts Connector currently presents 47 plans, a one from a separate set of 8 plans. Plan set order was counter- discussion of choice set size) [12]. balanced so half of the subjects chose from the 4-plan set first and In all six experiments, subjects were asked to imagine they were half chose from the 8-plan set first. Within each set of 4 and 8 choosing health insurance for a family of three – themselves, a plans, the display order of plans was also varied. In some experiments the number of visits or anticipated costs were varied partner, and one child – with an anticipated number of doctor visits and out-of-pocket health care costs over the next year. Each (described below). subject was required to choose one plan from a set of 4 plans and PLOS ONE | www.plosone.org 3 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges All studies shared certain features: All responses were collected Medicare.gov, provide such a tool. The present studies empha- online (see Table S1 for demographics and other details). To sized another important change designed to help diagnose the isolate the effect of making a choice from a misunderstanding of cause of poor performance: Plan attributes were drawn from an the basic mechanics of health insurance, each session included orthogonal experimental design, allowing us to estimate the weight explanations about insurance terms, such as premium, co-pay, and participants give to the three cost components, premiums, co- deductible, and required respondents to pass a comprehension test payments and deductibles. According to economic theory, these before proceeding (see methodological details in Methods S1 for costs should be approximately equally weighted since they all the content of these instructions and tests). Only those participants occur over the course of a year, and all contribute to the annual who passed this test were included in our analyses. Respondents cost of the policy. However, past research has indicated that some costs (usually deductibles) are overweighted while others, like viewed a table modeled after prototypes of exchanges (Figure 1) premiums are underweighted [4,13,14]. In addition, Experiment 4 and chose an insurance plan. In Experiment 1 and 2, all also simplified the choice by removing quality information for half components of prices resembled current prices and relationships of the respondents – this information was not diagnostic, since all among prices seen in existing and prototype exchanges. In options had the same total quality, and the choices made by addition, Experiments 1–2 varied, between respondents, the respondents confirmed this. number of visits, while Experiments 3–5 varied the level of out- of-pocket costs. For the sake of brevity, we will not discuss these The results, shown in the third and fourth columns of Figure 2 (A), are not markedly different. Again respondents chose the most results here. cost effective option less than half the time, and made large Experiment 1 provided a baseline measure of the proportion of financial errors. The unaided decision-makers averaged errors of people who choose the most cost-effective policy from 4 or 8 $611 in Experiment 3 and chose the correct option 32% of the options. Figure 2 shows the outcomes from all experiments. The top half of each bar, in blue, represents the proportion of correct time. Providing the calculators marginally helped but only in Experiment 4: Respondents provided with calculators chose the choices, and the bottom half, in red, plots the average dollar error, across respondents. We model all choices using a logistic model correct option 10.1% more often, and reduced the size of errors by with indicator variables for categorical variables, and an Analysis $216, but still were only correct 47% of the time and made mean of Variance to test significance for the error cost variable – please errors of $364. see Methods S1 for more details. The dashed line represents Why was performance so poor? Answering this question may expected choice quality by a random chooser. Panel A of Figure 2 suggest interventions. While the math alone is challenging, the shows a rather dramatic outcome: With 4 choice options, failure of the calculator to improve choice suggests that something respondents selected the best option only 42 percent of the time, else may be going on. Recall that past research shows that and made an average mistake of over $200 dollars. With eight deductibles may be overweighted [13–16]. If this is the case, options, they selected the correct option 21 percent of the time, a consumers may, arguably, have an incorrect notion of how figure not different than chance (p..05). deductibles contribute to overall cost. Figure 3 shows the weight Experiment 2 added monetary incentives: Selecting the most given to each price component in Experiment 4. The results show a strong and consistent bias, compared to the ideal of equal cost-effective policy increased payment by $1 and generated an entry to a lottery that paid $200 to one correct chooser – including weighting: Participants overweight the out-of-pocket costs and deductibles. Their improved performance with calculators is due, the lottery, the expected value of selecting each right option was in part, to reducing this bias, as illustrated by the red bar. In other $1.88, and performance was unrelated to time spent on the task. words, the presence of a calculator suggests that respondents came As can be seen in the next two bars of Figure 2 (A), incentives did closer to treating all dollars as having the same cost. not improve outcomes, and performance was close to chance. Is this task simply impossible? Experiment 5 used a very This failure might be due to individuals’ inability to perform the different population to see how highly trained, financially literate daunting calculations. One obvious intervention, used in Exper- individuals might do. We presented MBA students enrolled in a iments 3 and 4, involves the use of a cost-calculator stating the class on consumer finance with the same task as in Experiment 4. annual total cost. In fact, several existing web sites, including The average GMAT of students at this school was 716, and 59% of students came from consulting or financial services and related fields. As seen in the first column of Figure 2(B), they performed appreciably better, choosing the right option 73% of the time, and making an average mistake of $126. Their self-reports of how they accomplished the task are interesting: Forty percent reported using excel (this group performed quite well, selecting the correct option 85% of the time, and making an average error of only $47). This suggests that having both the right mental model and the ability to execute these calculations may be a basic requirement to make good choices. In Experiment 6, we explored the possibility that mental models in conjunction with different possible interventions would produce good performance by individuals who will be using the exchange. To ensure understanding, and encourage the use of the correct mental model, all conditions received a tutorial about computing the annual cost and completed a quiz requiring one correct choice. Figure 3. Premiums, deductibles, and co-payments, both We believe that this kind of just-in-time education might help both without calculator (blue) and with calculator (red). The decline aided and unaided choice, and further eliminate a lack of in odds of being chosen for each increase in $100 in annual cost for the knowledge (as opposed to computational complexity) as a barrier three cost components in experiments. doi:10.1371/journal.pone.0081521.g003 to better performance. We then compared this control condition PLOS ONE | www.plosone.org 4 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges to four different manipulations. An incentive group received a Conclusions more extreme and sophisticated incentive regime that penalized Our results present a bad news/good news story of particular respondents 10 cents for every $100 extra that was spent on importance. The bad news: Consumers left to their own devices insurance. We contrasted this to three choice architecture seem to make large errors when choosing health insurance, interventions. The first provided a calculator, explained what the suggesting that they will select options that are not cost-efficient calculator did, and tested that understanding. The second and they seem to be unaware of their failure. If consumers cannot provided a smart default that preselected the most cost effective identify cost-efficient plans, then the exchanges will not produce options given individuals’ usage. Individuals could, and did, competitive pressures on health plan costs, one of the main change that selection if desired. Finally, we combined defaults and advantages of relying upon choice and markets. It is possible that calculators. The presence of incentives and our choice architecture other factors, such as advertising and brokers may make the manipulations allowed us to compare the cost effectiveness of these market more or less competitive. The impact of such institutions is interventions. a question for further research. The last four bars in Figure 2(B), which average data over the The good news is that we have demonstrated that exchange number of options, show that the treatments vary widely in designers can improve consumers’ performance markedly through effectiveness. The controls, despite having received instruction and the use of just-in-time education, smart defaults, and cost tests of understanding, chose about as well as respondents in prior calculators. This list of potential design improvements is not experiments. The second bar indicates that incentives did not have exhaustive, and there are many other interventions that may a significant effect on outcomes, even though individuals in the improve choices. These include sorting by cost, the presence of incentive condition took 38% longer to make their decisions, a quality cues, or limiting the number of options to those that meet significant increase relative to controls. Calculators (with educa- criteria of cost-effectiveness. These suggestions are not without tion), in contrast, produced better decisions, having resulted in a precedent: In evaluations of Medicare Part D, Abaluck and significant decrease in the size of the loss and an increase in the Gruber [4] suggest that ‘‘restricting the choice set to the 3 lowest proportion correct. The smart default option had a similar effect, average cost options would have likely raised welfare for the as it reduced losses and increased the percentage correct. It is elders.’’ However, this limits consumer choice and we note that important to note that the performance of defaults is not simply some design features, such as calculators, improve outcomes by due to their mindless selection. First note that a significant making choice easier, without impinging upon consumer sover- proportion of people (21%) chose to not take the default by eignty. actively selecting another option. Second, those choosing the The results of these studies allow us to approximately estimate default option did take a significant amount of time to choose a the benefits of these kinds of choice architecture interventions. policy. Across the entire study, non-default choosers required These estimates should be treated with appropriate caution 443 seconds to complete the study, and choosers required because they are based on the particular set of policies used in 348 seconds. Concentrating on only the choice screen, default our studies. However, our control group in Experiment 6 made an choices took 58% and 65% as long as the no-default condition for average error of $533, roughly 10% of the cost of the cheapest the 4 and 8 option conditions, respectively. Finally, when policy, compared to an error of $77 when both the default option combined, the defaults and calculators seemed to complement and calculator were available, producing an estimated value to each other, leading to performance levels that are comparable to these features of $456 dollars per decision. At the individual level, those of the highly trained MBA students. This last result suggests, unaided choice is expensive: It represents about 1% of the income perhaps, that because calculators provide a justification for the of the proposed median buyers’ household income. But in the default, they increase the transparency of their selection, and aggregate, an error of $456 represents staggering sums: If 20 increases their adoption. It also suggests that providing just-in-time million individuals make choices using the exchanges, a figure education along with calculation and choice aids produces better suggested by Congressional Budget Office estimates, unaided performance. choice represents a cost to consumers of $9.12 billion dollars each While these interventions are effective, are they appreciated? year. Since almost all of these policies are subsidized through tax This is an important question about meta-cognition that has credits, good choice architecture would produce substantial important policy implications: If deciders are doing badly and savings to the federal budget and taxpayers. need help, do they realize it? When they get help, do they This sizable impact is more significant since the improvement is appreciate it? We asked respondents how confident they were of largely a function of psychological factors that can be implemented making the correct choice in Experiments 3, 4 and 6, using a 1–9 inexpensively by being built into the choice engines powering the point scale: While participants performed poorly, this was not exchanges. Clearly, further research identifying the best mix of reflected in their confidence ratings (mean rating 6.6, 6.75, and choice architecture tools in exchanges is both scientifically 7.6, respectively, in Experiment 3, 4, and the control condition in interesting and economically justified. While the success of the Experiment 6) and there was no correlation between these ratings health exchanges will depend, in part, on the provision of cost- and selecting the most cost effective plan (.09 averaged across these efficient products, it also will depend on the design of exchanges that three studies). It appears that individuals did not realize the need will allow consumers to identify the best choice that is a good fit to for these interventions. They also did not appreciate the effect of their needs. Ignoring the impact of choice architecture and the the interventions consistently: Calculators created a marginal psychological factors we examine could be an expensive mistake. increase in confidence (+.23 relative to control, p,.06); defaults did not (+.14, p..2). Finally, incentives did not increase Supporting Information performance, but they did increase effort and produced an Table S1 Demographics. unwarranted increase in confidence (+.34, p,.03). All told, the (DOCX) picture that emerges is that of overconfident decision-makers who do poorly and do not realize it, and who do not realize that Methods S1 Methods and Materials. decision-architecture helped. (DOCX) PLOS ONE | www.plosone.org 5 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges Acknowledgments Author Contributions Conceived and designed the experiments: EJ RH TB AB. Performed the We thank Margaret Lee and Jon Westfall for their assistance. We also experiments: EJ RH TB AB GT. Analyzed the data: EJ RH GT. Wrote the thank participants at the Sloan and Sage Consumer Finance Working Group and the Preference as Memories group for helpful comments at paper: EJ RH TB AB GT. presentations. Data from all experiments will be archived at www.plosone. org. References 1. Department of Health and Human Resources (2011) HHS and states move to 8. Ericson KM, Starc A (2012) Heuristics and heterogeneity in health insurance establish affordable insurance exchanges, give Americans the same insurance exchanges: Evidence from the Massachusetts connector. The American choices as members of congress. Available: http://www.hhs.gov/news/press/ Economic Review 102: 493–497. 9. Gigerenzer G, Todd PM, the ABC research group (1999) Simple heuristics that 2011pres/07/20110711a.html. Accessed 2012 Aug 1. make us smart. New York: Oxford University Press. 2. Alonso-Zaldivar R (2011) Health care overhaul debate now shifts to state. 10. Kaiser Family Foundation (2011) A profile of health insurance exchange Available: http://www.newsmax.com/Newsfront/HealthCareStates/2011/01/ enrollees. Available: http://www.kff.org/healthreform/8147.cfm. Accessed 22/id/383588. Accessed 2012 July 11. 2012 Aug 21. 3. Heiss F, McFadden D, Winter J (2010) Mind the gap! Consumer perceptions 11. Harrison GW, List JA (2004) Field experiments. Journal of Economic Literature and choices of Medicare Part D prescription drug plans. In Wise DA, editors. 42: 1009–1055. Research findings in the economics of aging. London: Chicago University Press. 12. Reutskaja E, Nagel R, Camerer CF, Rangel A (2011) Search dynamics in 413–381. consumer choice under time pressure: An eye-tracking study. The American 4. Abaluck J, Gruber J (2011) Heterogeneity in choice inconsistencies among the Economic Review 101: 900–926. elderly: Evidence from prescription drug plan choice. The American Economic 13. Johnson EJ, Hershey J, Meszaros J, Kunreuther H (1993) Framing, probability Review 101: 377–381. distortions, and insurance decisions. Journal of Risk and Uncertainty 7: 35–51. 5. Goldstein DG, Johnson EJ, Herrmann A, Heitmann M (2008) Nudge your 14. Sydnor J (2010) (Over) insuring modest risks. American Economic Journal: customers toward better choices. Harvard Business Review 86: 99–105. Applied Economics 2: 177–199. 6. Thaler RH, Sunstein CR (2008) Nudge: Improving decisions about health, 15. Kunreuther H, Pauly M (2005) Insurance decision-making and market behavior. wealth, and happiness. New Haven: Yale University Press. Foundations and trends in microeconomics 1: 63–127. 7. Johnson EJ, Shu SB, Dellaert BGC, Fox C, Goldstein DG, et el. (2012) Beyond 16. Barseghyan L, Prince J, Teitelbaum JC (2011) Are risk preferences stable across nudges: Tools of a choice architecture. Marketing Letters 23: 487–504. contexts? Evidence from insurance data. The American Economic Review 101: 591–631. PLOS ONE | www.plosone.org 6 December 2013 | Volume 8 | Issue 12 | e81521 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png PLoS ONE Pubmed Central

Can Consumers Make Affordable Care Affordable? The Value of Choice Architecture

PLoS ONE , Volume 8 (12) – Dec 18, 2013

Loading next page...
 
/lp/pubmed-central/can-consumers-make-affordable-care-affordable-the-value-of-choice-a4VVawjkek

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Pubmed Central
Copyright
© 2013 Johnson et al
ISSN
1932-6203
eISSN
1932-6203
DOI
10.1371/journal.pone.0081521
Publisher site
See Article on Publisher Site

Abstract

Tens of millions of people are currently choosing health coverage on a state or federal health insurance exchange as part of the Patient Protection and Affordable Care Act. We examine how well people make these choices, how well they think they do, and what can be done to improve these choices. We conducted 6 experiments asking people to choose the most cost- effective policy using websites modeled on current exchanges. Our results suggest there is significant room for improvement. Without interventions, respondents perform at near chance levels and show a significant bias, overweighting out-of-pocket expenses and deductibles. Financial incentives do not improve performance, and decision-makers do not realize that they are performing poorly. However, performance can be improved quite markedly by providing calculation aids, and by choosing a ‘‘smart’’ default. Implementing these psychologically based principles could save purchasers of policies and taxpayers approximately 10 billion dollars every year. Citation: Johnson EJ, Hassin R, Baker T, Bajger AT, Treuer G (2013) Can Consumers Make Affordable Care Affordable? The Value of Choice Architecture. PLoS ONE 8(12): e81521. doi:10.1371/journal.pone.0081521 Editor: Thomas Boraud, Centre national de la recherche scientifique, France Received March 28, 2013; Accepted October 14, 2013; Published December 18, 2013 Copyright:  2013 Johnson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This work was supported by the Alfred P. Sloan and Russell Sage Foundations (Grant number: 2011-5-12-ECON). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. * E-mail: ejj3@columbia.edu support decision-making. In this paper, we examine three related Introduction questions: Can people select the best policies? Do they know how Currently tens of millions of Americans, along with members of well they are doing? Does the design of the sites change their Congress and their staffs, are participating in a grand experiment performance? in consumer choice: They will select health insurance using a Our results suggest there is significant room to improve these marketplace or health insurance exchange operated by states and decisions. Without any intervention, respondents perform at near federal governments as part of the Patient Protection and chance levels and show a significant bias, overweighting out-of- Affordable Care Act. The success of these exchanges depends pocket costs and deductibles. Financial incentives do not improve upon two related premises: First that consumers will be able to performance, and decision-makers do not realize that they are select the best policy for their needs, and second that price performing badly. Without aids, only one population examined competition, driven by effective consumer choice, will lower here, Columbia MBA students, perform reasonably well at this prices. This hope is shared by divergent participants: Kathleen task. However, performance can be markedly improved by Sibelius, Secretary of Health and Human Services, and a providing calculation aids, and by choosing a ‘‘smart’’ default, Democrat, characterizes an exchange as ‘‘… a transparent, level raising the performance of ordinary respondents to that of the playing field, driving down costs; … giv[ing] individuals and small MBA students. businesses the same purchasing power as big businesses and a choice of plans to fit their needs.’’ [1] Bill Frist, a physician and Prior Research former Republican Senate Majority Leader, argues ‘‘State The quality of choices on prior health insurance exchanges has exchanges are good from a conservative standpoint because they been, at best, mixed. For example, when examining the exchanges involve consumer choice and markets.’’ [2]. implementing Medicare Part D, a prescription drug plan for These premises are critical not only to the new exchanges, but seniors, Heiss, McFadden, and Winter [3] conclude, ‘‘consumers also for all government administered health insurance markets and are likely to have difficulty choosing among plans to fine tune their for the efficiency of privately provided benefit choices – both prescription drug coverage.’’ Abaluck and Gruber [4] find that health exchanges and employer sponsored insurance center on only 12.2% of seniors pick the most cost-effective plan. consumer choice in finding plans that are cost-effective and While the economic analysis of choice suggests that issues appropriate for consumer needs and both include many design surrounding incentives and information may determine success, a decisions that will affect choice. Yet a large literature in psychology more psychological analysis suggests that good decisions depend, suggests that these premises may not be realistic, since, as we shall critically, on subtle elements of how the choices are presented to see, these exchanges may not provide a helpful choice architecture to PLOS ONE | www.plosone.org 1 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges the consumer, as described in an evolving literature on choice see Methods S1 for a thorough explanation of the experimental procedure. architecture [5–7]. Designing an exchange involves many design decisions including specifying the number and kind of options and Even in this simplified version of health insurance choice, the attributes offered, determining the arrangement of options and the process of selecting the most cost-effective health insurance is not format and order of attributes, and selecting default options and trivial, consumers must, for each plan: computational aids. 1. Consider the total premiums for the year, The Massachusetts ‘‘Connector,’’ an exchange operating since 2. Combine the copayments and the expected number of visits, 2006, illustrates the impact of choice architecture: Before late 2009, the Connector simultaneously presented 25 plans from 6 and insurance providers. In 2009, plans were reorganized into 3 tiers of 3. Include the minimum of the deductible and their out-of-pocket coverage, categorized by premiums and out-of-pocket costs. costs. Consumers first chose one of these levels and then viewed a For equal monthly premiums this is (12*Monthly Premium) + smaller set of 6 standardized plans within a level. Work by Ericson and Starc [8] shows that this simple change markedly altered (N of visits * Copay) + min(Out-of-Pocket Costs, Deductible). Adding risk considerations, while undoubtedly important, would behavior: Consumers were increasingly sensitive to premium costs and out-of-pocket costs, changing market shares for some carriers make these calculations even more difficult, thus making performance worse, not better than we observe, and perhaps by a factor of 2. make our interventions more effective. Thus, the advent of health exchanges presents a challenge: The The reader might consider selecting the most cost effective plan choice could be daunting for consumers, resulting in suboptimal in Figure 1, assuming, as did respondents in one of our choices of policies that provide the wrong features or are too experimental conditions, that he or she will make 9 doctor visits expensive. We are interested in how a prudent design of health exchanges based on psychological research could improve choice. and incur $900 in out-of-pocket costs in the upcoming year. This calculation might seem difficult, but some would argue that there We are also interested in a parallel question: Do people know if they are making good decisions? This is important because if might be heuristic strategies that perform well [9]. Yet we feel that there are two reasons for concern: First, users of these exchanges people know that they are not doing well, they could seek assistance, potentially remedying their poor performance. If will be largely unfamiliar with selecting health insurance – since many, 97% according to some estimates [7], will be buying health people are unaware of their inadequate performance, simply insurance for the first time and may lack experience and relevant providing access to assistance will not improve their decision- knowledge – and will not be highly educated (seventy-seven making. percent will have a High School diploma or less) [10]. Second, this is an economically significant decision for these households: Even Methods: Choosing health insurance with subsidies, premiums will represent between 4 and 9.5% of the When choosing insurance, consumers face two tasks. The first, modest median income of $48,529 for a family of 4 [10]. which we do not examine, is to estimate their expected usage and Consequently, mistakes may have large economic consequences. out-of-pocket expenses for the upcoming year, and to consider the uncertainty around these quantities as a choice under risk. The Results and Discussion: Can Consumers Choose second is to select the right plan given their expected usage. The Right Plan? Our studies focused on people’s ability to select cost-effective policies and remove risk and usage prediction considerations. We examined consumers’ decision-making abilities and condi- While economists analyze insurance choice by examining uncer- tions that might facilitate better decisions in a series of six framed tainty, risk, and asymmetric information, we investigated the field experiments [11], all but one using participants with impact of psychological variables such as calculation costs as a demographics similar to those projected to use the exchanges. In major barrier to better choices. We examined a simplified version addition to specifying the number of doctor visits one would make of the health insurance choice that allowed us to assess the and the out-of-pocket costs one would incur in a given year, we performance of choice architecture interventions, much like a also limited the number of plans available to either 4 or 8, a figure wind tunnel might be used to evaluate candidate airplane designs – markedly lower than the number to be used in future exchanges Figure 1. A decision display used in Experiment 4. Respondents saw either 8 (pictured) or 4 options. doi:10.1371/journal.pone.0081521.g001 PLOS ONE | www.plosone.org 2 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges Figure 2. The percentage of choices of the most cost effective option and respondents’ average error. The top half of each bar, in blue, represents the proportion of correct choices, and the bottom half, below the zero line in red, plots the average dollar error, across respondents. A dashed line for each condition represents the performance of a random chooser, and the error bars represent 95% confidence intervals. Darker shades denote the provision of calculators. Panel (A) represents the results of Experiments 1–4 collapsing across other manipulations (see SM). Panel (B) represents the results of a sample of highly educated MBA students (Experiment 5), and of individuals from the target population, when given different choice architecture interventions. For (b) the random response threshold ($1264) exceeds the lower limit of the graph. doi:10.1371/journal.pone.0081521.g002 (e.g., the Massachusetts Connector currently presents 47 plans, a one from a separate set of 8 plans. Plan set order was counter- discussion of choice set size) [12]. balanced so half of the subjects chose from the 4-plan set first and In all six experiments, subjects were asked to imagine they were half chose from the 8-plan set first. Within each set of 4 and 8 choosing health insurance for a family of three – themselves, a plans, the display order of plans was also varied. In some experiments the number of visits or anticipated costs were varied partner, and one child – with an anticipated number of doctor visits and out-of-pocket health care costs over the next year. Each (described below). subject was required to choose one plan from a set of 4 plans and PLOS ONE | www.plosone.org 3 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges All studies shared certain features: All responses were collected Medicare.gov, provide such a tool. The present studies empha- online (see Table S1 for demographics and other details). To sized another important change designed to help diagnose the isolate the effect of making a choice from a misunderstanding of cause of poor performance: Plan attributes were drawn from an the basic mechanics of health insurance, each session included orthogonal experimental design, allowing us to estimate the weight explanations about insurance terms, such as premium, co-pay, and participants give to the three cost components, premiums, co- deductible, and required respondents to pass a comprehension test payments and deductibles. According to economic theory, these before proceeding (see methodological details in Methods S1 for costs should be approximately equally weighted since they all the content of these instructions and tests). Only those participants occur over the course of a year, and all contribute to the annual who passed this test were included in our analyses. Respondents cost of the policy. However, past research has indicated that some costs (usually deductibles) are overweighted while others, like viewed a table modeled after prototypes of exchanges (Figure 1) premiums are underweighted [4,13,14]. In addition, Experiment 4 and chose an insurance plan. In Experiment 1 and 2, all also simplified the choice by removing quality information for half components of prices resembled current prices and relationships of the respondents – this information was not diagnostic, since all among prices seen in existing and prototype exchanges. In options had the same total quality, and the choices made by addition, Experiments 1–2 varied, between respondents, the respondents confirmed this. number of visits, while Experiments 3–5 varied the level of out- of-pocket costs. For the sake of brevity, we will not discuss these The results, shown in the third and fourth columns of Figure 2 (A), are not markedly different. Again respondents chose the most results here. cost effective option less than half the time, and made large Experiment 1 provided a baseline measure of the proportion of financial errors. The unaided decision-makers averaged errors of people who choose the most cost-effective policy from 4 or 8 $611 in Experiment 3 and chose the correct option 32% of the options. Figure 2 shows the outcomes from all experiments. The top half of each bar, in blue, represents the proportion of correct time. Providing the calculators marginally helped but only in Experiment 4: Respondents provided with calculators chose the choices, and the bottom half, in red, plots the average dollar error, across respondents. We model all choices using a logistic model correct option 10.1% more often, and reduced the size of errors by with indicator variables for categorical variables, and an Analysis $216, but still were only correct 47% of the time and made mean of Variance to test significance for the error cost variable – please errors of $364. see Methods S1 for more details. The dashed line represents Why was performance so poor? Answering this question may expected choice quality by a random chooser. Panel A of Figure 2 suggest interventions. While the math alone is challenging, the shows a rather dramatic outcome: With 4 choice options, failure of the calculator to improve choice suggests that something respondents selected the best option only 42 percent of the time, else may be going on. Recall that past research shows that and made an average mistake of over $200 dollars. With eight deductibles may be overweighted [13–16]. If this is the case, options, they selected the correct option 21 percent of the time, a consumers may, arguably, have an incorrect notion of how figure not different than chance (p..05). deductibles contribute to overall cost. Figure 3 shows the weight Experiment 2 added monetary incentives: Selecting the most given to each price component in Experiment 4. The results show a strong and consistent bias, compared to the ideal of equal cost-effective policy increased payment by $1 and generated an entry to a lottery that paid $200 to one correct chooser – including weighting: Participants overweight the out-of-pocket costs and deductibles. Their improved performance with calculators is due, the lottery, the expected value of selecting each right option was in part, to reducing this bias, as illustrated by the red bar. In other $1.88, and performance was unrelated to time spent on the task. words, the presence of a calculator suggests that respondents came As can be seen in the next two bars of Figure 2 (A), incentives did closer to treating all dollars as having the same cost. not improve outcomes, and performance was close to chance. Is this task simply impossible? Experiment 5 used a very This failure might be due to individuals’ inability to perform the different population to see how highly trained, financially literate daunting calculations. One obvious intervention, used in Exper- individuals might do. We presented MBA students enrolled in a iments 3 and 4, involves the use of a cost-calculator stating the class on consumer finance with the same task as in Experiment 4. annual total cost. In fact, several existing web sites, including The average GMAT of students at this school was 716, and 59% of students came from consulting or financial services and related fields. As seen in the first column of Figure 2(B), they performed appreciably better, choosing the right option 73% of the time, and making an average mistake of $126. Their self-reports of how they accomplished the task are interesting: Forty percent reported using excel (this group performed quite well, selecting the correct option 85% of the time, and making an average error of only $47). This suggests that having both the right mental model and the ability to execute these calculations may be a basic requirement to make good choices. In Experiment 6, we explored the possibility that mental models in conjunction with different possible interventions would produce good performance by individuals who will be using the exchange. To ensure understanding, and encourage the use of the correct mental model, all conditions received a tutorial about computing the annual cost and completed a quiz requiring one correct choice. Figure 3. Premiums, deductibles, and co-payments, both We believe that this kind of just-in-time education might help both without calculator (blue) and with calculator (red). The decline aided and unaided choice, and further eliminate a lack of in odds of being chosen for each increase in $100 in annual cost for the knowledge (as opposed to computational complexity) as a barrier three cost components in experiments. doi:10.1371/journal.pone.0081521.g003 to better performance. We then compared this control condition PLOS ONE | www.plosone.org 4 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges to four different manipulations. An incentive group received a Conclusions more extreme and sophisticated incentive regime that penalized Our results present a bad news/good news story of particular respondents 10 cents for every $100 extra that was spent on importance. The bad news: Consumers left to their own devices insurance. We contrasted this to three choice architecture seem to make large errors when choosing health insurance, interventions. The first provided a calculator, explained what the suggesting that they will select options that are not cost-efficient calculator did, and tested that understanding. The second and they seem to be unaware of their failure. If consumers cannot provided a smart default that preselected the most cost effective identify cost-efficient plans, then the exchanges will not produce options given individuals’ usage. Individuals could, and did, competitive pressures on health plan costs, one of the main change that selection if desired. Finally, we combined defaults and advantages of relying upon choice and markets. It is possible that calculators. The presence of incentives and our choice architecture other factors, such as advertising and brokers may make the manipulations allowed us to compare the cost effectiveness of these market more or less competitive. The impact of such institutions is interventions. a question for further research. The last four bars in Figure 2(B), which average data over the The good news is that we have demonstrated that exchange number of options, show that the treatments vary widely in designers can improve consumers’ performance markedly through effectiveness. The controls, despite having received instruction and the use of just-in-time education, smart defaults, and cost tests of understanding, chose about as well as respondents in prior calculators. This list of potential design improvements is not experiments. The second bar indicates that incentives did not have exhaustive, and there are many other interventions that may a significant effect on outcomes, even though individuals in the improve choices. These include sorting by cost, the presence of incentive condition took 38% longer to make their decisions, a quality cues, or limiting the number of options to those that meet significant increase relative to controls. Calculators (with educa- criteria of cost-effectiveness. These suggestions are not without tion), in contrast, produced better decisions, having resulted in a precedent: In evaluations of Medicare Part D, Abaluck and significant decrease in the size of the loss and an increase in the Gruber [4] suggest that ‘‘restricting the choice set to the 3 lowest proportion correct. The smart default option had a similar effect, average cost options would have likely raised welfare for the as it reduced losses and increased the percentage correct. It is elders.’’ However, this limits consumer choice and we note that important to note that the performance of defaults is not simply some design features, such as calculators, improve outcomes by due to their mindless selection. First note that a significant making choice easier, without impinging upon consumer sover- proportion of people (21%) chose to not take the default by eignty. actively selecting another option. Second, those choosing the The results of these studies allow us to approximately estimate default option did take a significant amount of time to choose a the benefits of these kinds of choice architecture interventions. policy. Across the entire study, non-default choosers required These estimates should be treated with appropriate caution 443 seconds to complete the study, and choosers required because they are based on the particular set of policies used in 348 seconds. Concentrating on only the choice screen, default our studies. However, our control group in Experiment 6 made an choices took 58% and 65% as long as the no-default condition for average error of $533, roughly 10% of the cost of the cheapest the 4 and 8 option conditions, respectively. Finally, when policy, compared to an error of $77 when both the default option combined, the defaults and calculators seemed to complement and calculator were available, producing an estimated value to each other, leading to performance levels that are comparable to these features of $456 dollars per decision. At the individual level, those of the highly trained MBA students. This last result suggests, unaided choice is expensive: It represents about 1% of the income perhaps, that because calculators provide a justification for the of the proposed median buyers’ household income. But in the default, they increase the transparency of their selection, and aggregate, an error of $456 represents staggering sums: If 20 increases their adoption. It also suggests that providing just-in-time million individuals make choices using the exchanges, a figure education along with calculation and choice aids produces better suggested by Congressional Budget Office estimates, unaided performance. choice represents a cost to consumers of $9.12 billion dollars each While these interventions are effective, are they appreciated? year. Since almost all of these policies are subsidized through tax This is an important question about meta-cognition that has credits, good choice architecture would produce substantial important policy implications: If deciders are doing badly and savings to the federal budget and taxpayers. need help, do they realize it? When they get help, do they This sizable impact is more significant since the improvement is appreciate it? We asked respondents how confident they were of largely a function of psychological factors that can be implemented making the correct choice in Experiments 3, 4 and 6, using a 1–9 inexpensively by being built into the choice engines powering the point scale: While participants performed poorly, this was not exchanges. Clearly, further research identifying the best mix of reflected in their confidence ratings (mean rating 6.6, 6.75, and choice architecture tools in exchanges is both scientifically 7.6, respectively, in Experiment 3, 4, and the control condition in interesting and economically justified. While the success of the Experiment 6) and there was no correlation between these ratings health exchanges will depend, in part, on the provision of cost- and selecting the most cost effective plan (.09 averaged across these efficient products, it also will depend on the design of exchanges that three studies). It appears that individuals did not realize the need will allow consumers to identify the best choice that is a good fit to for these interventions. They also did not appreciate the effect of their needs. Ignoring the impact of choice architecture and the the interventions consistently: Calculators created a marginal psychological factors we examine could be an expensive mistake. increase in confidence (+.23 relative to control, p,.06); defaults did not (+.14, p..2). Finally, incentives did not increase Supporting Information performance, but they did increase effort and produced an Table S1 Demographics. unwarranted increase in confidence (+.34, p,.03). All told, the (DOCX) picture that emerges is that of overconfident decision-makers who do poorly and do not realize it, and who do not realize that Methods S1 Methods and Materials. decision-architecture helped. (DOCX) PLOS ONE | www.plosone.org 5 December 2013 | Volume 8 | Issue 12 | e81521 Valuing Choice Architecture on Health Exchanges Acknowledgments Author Contributions Conceived and designed the experiments: EJ RH TB AB. Performed the We thank Margaret Lee and Jon Westfall for their assistance. We also experiments: EJ RH TB AB GT. Analyzed the data: EJ RH GT. Wrote the thank participants at the Sloan and Sage Consumer Finance Working Group and the Preference as Memories group for helpful comments at paper: EJ RH TB AB GT. presentations. Data from all experiments will be archived at www.plosone. org. References 1. Department of Health and Human Resources (2011) HHS and states move to 8. Ericson KM, Starc A (2012) Heuristics and heterogeneity in health insurance establish affordable insurance exchanges, give Americans the same insurance exchanges: Evidence from the Massachusetts connector. The American choices as members of congress. Available: http://www.hhs.gov/news/press/ Economic Review 102: 493–497. 9. Gigerenzer G, Todd PM, the ABC research group (1999) Simple heuristics that 2011pres/07/20110711a.html. Accessed 2012 Aug 1. make us smart. New York: Oxford University Press. 2. Alonso-Zaldivar R (2011) Health care overhaul debate now shifts to state. 10. Kaiser Family Foundation (2011) A profile of health insurance exchange Available: http://www.newsmax.com/Newsfront/HealthCareStates/2011/01/ enrollees. Available: http://www.kff.org/healthreform/8147.cfm. Accessed 22/id/383588. Accessed 2012 July 11. 2012 Aug 21. 3. Heiss F, McFadden D, Winter J (2010) Mind the gap! Consumer perceptions 11. Harrison GW, List JA (2004) Field experiments. Journal of Economic Literature and choices of Medicare Part D prescription drug plans. In Wise DA, editors. 42: 1009–1055. Research findings in the economics of aging. London: Chicago University Press. 12. Reutskaja E, Nagel R, Camerer CF, Rangel A (2011) Search dynamics in 413–381. consumer choice under time pressure: An eye-tracking study. The American 4. Abaluck J, Gruber J (2011) Heterogeneity in choice inconsistencies among the Economic Review 101: 900–926. elderly: Evidence from prescription drug plan choice. The American Economic 13. Johnson EJ, Hershey J, Meszaros J, Kunreuther H (1993) Framing, probability Review 101: 377–381. distortions, and insurance decisions. Journal of Risk and Uncertainty 7: 35–51. 5. Goldstein DG, Johnson EJ, Herrmann A, Heitmann M (2008) Nudge your 14. Sydnor J (2010) (Over) insuring modest risks. American Economic Journal: customers toward better choices. Harvard Business Review 86: 99–105. Applied Economics 2: 177–199. 6. Thaler RH, Sunstein CR (2008) Nudge: Improving decisions about health, 15. Kunreuther H, Pauly M (2005) Insurance decision-making and market behavior. wealth, and happiness. New Haven: Yale University Press. Foundations and trends in microeconomics 1: 63–127. 7. Johnson EJ, Shu SB, Dellaert BGC, Fox C, Goldstein DG, et el. (2012) Beyond 16. Barseghyan L, Prince J, Teitelbaum JC (2011) Are risk preferences stable across nudges: Tools of a choice architecture. Marketing Letters 23: 487–504. contexts? Evidence from insurance data. The American Economic Review 101: 591–631. PLOS ONE | www.plosone.org 6 December 2013 | Volume 8 | Issue 12 | e81521

Journal

PLoS ONEPubmed Central

Published: Dec 18, 2013

References