Field Experiments GLENN W.HARRISON and JOHN A.LISTI 1.Introduction experimental environment.We do not see In some sense every empirical researcher is the notion of a "sterile environment"as a Lreporting the results of an experiment. negative,provided one recognizes its role in Every rese rcher who behaves as if an exoge. the research discovery process.In one nous variable varies independently of an error sense,that sterility allows us to see in crisp term effectively views their data as coming relief the effects of exogenous treatments on from an experiment.In some cases this belief behavior.However,lab experiments in isola is a matter of a priori judgement;in some tion are necessarily limited in relevance for cases it is based on auxiliary evidence and predicting field behavior,unless one wants inference;and in some cases it is built into the to insist a priori that those aspects of eco design of the data collection process.But the nomic behavior under study are distinction is not always as bright and clear Testing that assumption is a rec rring difficul- Rather.we see the beauty of lab appliedc ometricians.and the search ments within a br ty for context-when sha th fiel data,they pe tor mualify as truly pe and m under study.Similarly.the of ate 1 of explicit ime th ential for constructing rather thar n in the iting subject the tha b82ercm pd anc contex at ha bstract between thes approa empiric cience.By examining the nature of field exp ments,we seek to make it a common groun tween researchers field experim nts from the perspective of the sterility of the laboratory cely he and CcprociyareotienCeoeounmdedltwih】 one undertak Alvin Rot d Re Michael Malouf (1)de nonstrate how the use and with less ation of one kindg of th eas Ortmanr othe kind (the ab to make 1009 This content downl
Journal of Economic Literature Vol. XLII (December 2004) pp. 1009-1055 Field Experiments GLENN W. HARRISON and JOHN A. LIST1 1. Introduction n some sense every empirical researcher is reporting the results of an experiment. Every researcher who behaves as if an exoge- nous variable varies independently of an error term effectively views their data as coming from an experiment. In some cases this belief is a matter of a priori judgement; in some cases it is based on auxiliary evidence and inference; and in some cases it is built into the design of the data collection process. But the distinction is not always as bright and clear. Testing that assumption is a recurring difficul- ty for applied econometricians, and the search always continues for variables that might bet- ter qualify as truly exogenous to the process under study. Similarly, the growing popularity of explicit experimental methods arises in large part from the potential for constructing the proper counterfactual. Field experiments provide a meeting ground between these two broad approaches to empirical economic science. By examining the nature of field experiments, we seek to make it a common ground between researchers. We approach field experiments from the perspective of the sterility of the laboratory 1 Harrison: Department of Economics, College of Business Administration, University of Central Florida; List: Department of Agricultural and Resource Economics and Department of Economics, University of Maryland, and NBER. We are grateful to Stephen Burks, Colin Camerer, Jeffrey Carpenter, Shelby Gerking, R. Mark Isaac, Alan Krueger, John McMillan, Andreas Ortmann, Charles Plott, David Reiley, E. Elisabet Rutstrom, Nathaniel Wilcox, and the referees for generous comments. experimental environment. We do not see the notion of a "sterile environment" as a negative, provided one recognizes its role in the research discovery process. In one sense, that sterility allows us to see in crisp relief the effects of exogenous treatments on behavior. However, lab experiments in isola- tion are necessarily limited in relevance for predicting field behavior, unless one wants to insist a priori that those aspects of eco- nomic behavior under study are perfectly general in a sense that we will explain. Rather, we see the beauty of lab experi- ments within a broader context-when they are combined with field data, they permit sharper and more convincing inference.2 In search of greater relevance, experi- mental economists are recruiting subjects in the field rather than in the classroom, using field goods rather than induced valuations, and using field context rather than abstract 2 When we talk about combining lab and field data, we do not just mean a summation of conclusions. Instead, we have in mind the two complementing each other in some functional way, much as one might conduct several lab experiments in order to tease apart potential confounds. For example, James Cox (2004) demonstrates nicely how "trust" and "reciprocity" are often confounded with "other regarding preferences," and can be better identified sep- arately if one undertakes several types of experiments with the same population. Similarly, Alvin Roth and Michael Malouf (1979) demonstrate how the use of dollar payoffs can confound tests of cooperative game theory with less information of one kind (knowledge of the utili- ty function of the other player), and more information of another kind (the ability to make interpersonal compar- isons of monetary gain), than is usually assumed in the leading theoretical prediction. 1009 This content downloaded from 218.106.182.180 on Sat, 11 Jun 2016 06:18:54 UTC All use subject to http://about.jstor.org/terms
1010 Journal of Economic Literature,Vol.XLII(December 2004) terminology in instructions.3 We argue that Our second p there is something methodologically funda- acteristi be found mental behind this trend.Field experiments in varying. correlat egrees in lab differ from laboratory experiments in many ment nus,many of the characteristics that ways.Although it is tempting to view field people identify with f d experiments are exr eriments as simply less controlled variants not only found in field experiments,and of laboratory experiments,we argue that to do shoul not be used to differentiate them so would be to seriously mischar cterize them from lab experiments What passes for"control"in laboratory expe Our third point,following from the first iments might in fact be cisely the two,is that there is much to learn from field if it is artificial to the subject or contex ptof the experiments when returning to the lab.The task In the end we field unexpected behaviors that occur when one loosens control in the field are often indica- tradi品onal laborat ts entary to tors of key features of the economic transac Ou ecting the tion that have been neglected in the lab.Thus field experiments can help one design better what ca an lab experiments,and have a methodological role ense t quite apart from their complementarity subject in at a substantive level where the ject does not perceive any of In section 2 we offer a typology of field s in the liter the contro s as being unna and there is no deception being practiced.At first blush,the c atur ifying the acteris th idea that one can observe subjects in a natural logy to b ette setting and yet have controls might seem rimen contradictory,but we will argue that it is not. ntify diff nts We e do not propose We explain thi jargon from experimental economics ents s someth else,but ta set of criteria that one wou expect to see in varying degrees in a field experment.we propos actors that can be used to determine the field context of an experiment:the nature of the subject pool the nature of the information that the sub iects bring to the task.the nature of the com traditional lecture. modity,the nature of the task or trading rules other features ofthe applied,the nature of the stakes,and the environment in which the subjects operate Having identified what defines a field exner iment.in section 3 we put experiments in ective.as Along s nomists can identify treatment effects.This serves to remind us king-Reiley()Camerer( S)pla want control and internal validity in all such an not the sst ntemet-b hte field ments.In ctions 4 through and weakn major types of field experiment
Journal of Economic Literature, Vol. XLII (December 2004) terminology in instructions.3 We argue that there is something methodologically funda- mental behind this trend. Field experiments differ from laboratory experiments in many ways. Although it is tempting to view field experiments as simply less controlled variants of laboratory experiments, we argue that to do so would be to seriously mischaracterize them. What passes for "control" in laboratory exper- iments might in fact be precisely the opposite if it is artificial to the subject or context of the task. In the end, we see field experiments as being methodologically complementary to traditional laboratory experiments.4 Our primary point is that dissecting the characteristics of field experiments helps define what might be better called an ideal experiment, in the sense that one is able to observe a subject in a controlled setting but where the subject does not perceive any of the controls as being unnatural and there is no deception being practiced. At first blush, the idea that one can observe subjects in a natural setting and yet have controls might seem contradictory, but we will argue that it is not.5 3 We explain this jargon from experimental economics below. 4 This view is hardly novel: for example, in decision research, Robert Winkler and Allan Murphy (1973) pro- vide an excellent account of the difficulties of reconciling suboptimal probability assessments in artefactual laborato- ry settings with field counterparts, as well as the limitations of applying inferences from laboratory data to the field. Imagine a classroom setting in which the class breaks up into smaller tutorial groups. In some groups a video cov- ering certain material is presented, in another group a free discussion is allowed, and in another group there is a more traditional lecture. Then the scores of the students in each group are examined after they have taken a common exam. Assuming that all of the other features of the experiment are controlled, such as which student gets assigned to which group, this experiment would not seem unnatural to the sub- jects. They are all students doing what comes naturally to students, and these three teaching alternatives are each stan- dardly employed. Along similar lines in economics, albeit with simpler technology and less control than one might like, see Edward Duddy (1924). For recent novel examples in the economics literature, see Colin Camerer (1998) and David Lucking-Reiley (1999). Camerer (1998) places bets at a race track to examine if asset markets can be manipulated, while Lucking-Reiley (1999) uses internet-based auctions in a pre- existing market with an unknown number of participating bidders to test the theory of revenue equivalence between four major single-unit auction formats. Our second point is that many of the char- acteristics of field experiments can be found in varying, correlated degrees in lab experi- ments. Thus, many of the characteristics that people identify with field experiments are not only found in field experiments, and should not be used to differentiate them from lab experiments. Our third point, following from the first two, is that there is much to learn from field experiments when returning to the lab. The unexpected behaviors that occur when one loosens control in the field are often indica- tors of key features of the economic transac- tion that have been neglected in the lab. Thus, field experiments can help one design better lab experiments, and have a methodological role quite apart from their complementarity at a substantive level. In section 2 we offer a typology of field experiments in the literature, identifying the key characteristics defining the species. We suggest some terminology to better identify different types of field experiments, or more accurately to identify different characteris- tics of field experiments. We do not propose a bright line to define some experiments as field experiments and others as something else, but a set of criteria that one would expect to see in varying degrees in a field experiment. We propose six factors that can be used to determine the field context of an experiment: the nature of the subject pool, the nature of the information that the sub- jects bring to the task, the nature of the com- modity, the nature of the task or trading rules applied, the nature of the stakes, and the environment in which the subjects operate. Having identified what defines a field exper- iment, in section 3 we put experiments in general into methodological perspective, as one of the ways that economists can identify treatment effects. This serves to remind us why we want control and internal validity in all such analyses, whether or not they consti- tute field experiments. In sections 4 through 6 we describe strengths and weaknesses of the broad types of field experiments. Our 1010 This content downloaded from 218.106.182.180 on Sat, 11 Jun 2016 06:18:54 UTC All use subject to http://about.jstor.org/terms
Harrison and List:Field Experiments 1011 literature review is necessarily selective. material,language,animal,etc.,and not in although List(2004d)offers a more complete the laboratory.study.or office."This orients bibliography. us to think of the natural environment of the In sections 7 and 8 we review two types of different components of an experiment. experiments that may be contrasted with It is important to identify what factors ideal field experiments.One is called a social make up a field experiment so that we can experiment.in the sense that it is a deliber nally identify what factors drive ate part of social policy by the g results in different e periments.To provide eriments involve deliberate. a direct example of the type of problem that domized changes in the manner in which motivated t when List (2001)obtains nt ram is imple nented have he rog results in a field They me the coun mer ts of Ro ald e nd the de Harris and Ia nation.Thei nd La do given thei aylor (1999,wha and there explains h rom a p ons from the par o the de the des of field ent subje h tural experiment ubjects with experi The ide ais to re som vent tha use of private sports-card s as the unde natural ens to have ng c ead of an environme some of acteristics of a field experi tal public good the use of streamline ment.These can be attractive sources of data instructions,the less-intrusive experimental on large-scale economic transactions but methods,mundane experimenter effects, usually at some cost due to the lack of con is it some combination of these and similar trol.forcing the researcher to make certain identification assumptions. mine the role of "controls"in differer Finally,in section 9 we briefly examine related types of experiments of the mind.In one case these are the "thought experi- n the free action。 o hold sw ments"of theorists and statisticians and in and inte the other they are the "neuro-economics periments"provided by technology.The objective is simply to identify how they differ quip and where of xperiments we consider But w Gt in es d to be sor 2.Defining Field Experiment than it would othe the ing othet T y t to be sitive,when studyin from There are several spect.The reason is that the e the g it up ir the dicti ary.An tha it is that you want the word labe re is th The Ox rd En glish Dict ary (Seco T nt is y th in the "th Edition)defines the word"field in the fol ory of the lowing manner: Used attributively to ot auto denote an investigation,study,etc.,carried out in the natural environment of a given to the optimum This content downle
Harrison and List: Field Experiments literature review is necessarily selective, although List (2004d) offers a more complete bibliography. In sections 7 and 8 we review two types of experiments that may be contrasted with ideal field experiments. One is called a social experiment, in the sense that it is a deliber- ate part of social policy by the government. Social experiments involve deliberate, ran- domized changes in the manner in which some government program is implemented. They have become popular in certain areas, such as employment schemes and the detec- tion of discrimination. Their disadvantages have been well documented, given their political popularity, and there are several important methodological lessons from those debates for the design of field experiments. The other is called a "natural experiment." The idea is to recognize that some event that naturally occurs in the field happens to have some of the characteristics of a field experi- ment. These can be attractive sources of data on large-scale economic transactions, but usually at some cost due to the lack of con- trol, forcing the researcher to make certain identification assumptions. Finally, in section 9 we briefly examine related types of experiments of the mind. In one case these are the "thought experi- ments" of theorists and statisticians, and in the other they are the "neuro-economics experiments" provided by technology. The objective is simply to identify how they differ from other types of experiments we consider, and where they fit in. 2. Defining Field Experiments There are several ways to define words. One is to ascertain the formal definition by looking it up in the dictionary. Another is to identify what it is that you want the word- label to differentiate. The Oxford English Dictionary (Second Edition) defines the word "field" in the fol- lowing manner: "Used attributively to denote an investigation, study, etc., carried out in the natural environment of a given material, language, animal, etc., and not in the laboratory, study, or office." This orients us to think of the natural environment of the different components of an experiment.6 It is important to identify what factors make up a field experiment so that we can functionally identify what factors drive results in different experiments. To provide a direct example of the type of problem that motivated us, when List (2001) obtains results in a field experiment that differ from the counterpart lab experiments of Ronald Cummings, Glenn Harrison, and Laura Osborne (1995) and Cummings and Laura Taylor (1999), what explains the difference? Is it the use of data from a particular market whose participants have selected into the market instead of student subjects; the use of subjects with experience in related tasks; the use of private sports-cards as the under- lying commodity instead of an environmen- tal public good; the use of streamlined instructions, the less-intrusive experimental methods, mundane experimenter effects, or is it some combination of these and similar 6 If we are to examine the role of "controls" in different experimental settings, it is appropriate that this word also be defined carefully. The OED (2nd ed.) defines the verb "control" in the following manner: "To exercise restraint or direction upon the free action of; to hold sway over, exer- cise power or authority over; to dominate, command." So the word means something more active and intervention- ist than is suggested by its colloquial clinical usage. Control can include such mundane things as ensuring ster- ile equipment in a chemistry lab, to restrain the free flow of germs and unwanted particles that might contaminate some test. But when controls are applied to human behav- ior, we are reminded that someone's behavior is being restrained to be something other than it would otherwise be if the person were free to act. Thus we are immediate- ly on alert to be sensitive, when studying responses from a controlled experiment, to the possibility that behavior is unusual in some respect. The reason is that the very con- trol that defines the experiment may be putting the sub- ject on an artificial margin. Even if behavior on that margin is not different than it would be without the con- trol, there is the possibility that constraints on one margin may induce effects on behavior on unconstrained margins. This point is exactly the same as the one made in the "the- ory of the second best" in public policy. If there is some immutable constraint on one of the margins defining an optimum, it does not automatically follow that removing a constraint on another margin will move the system closer to the optimum. 1011 This content downloaded from 218.106.182.180 on Sat, 11 Jun 2016 06:18:54 UTC All use subject to http://about.jstor.org/terms
1019 Journal of Economic Literature,Vol.XLII(December 2004) differences?We believe field experiments (McKinley Blackburn. Harrison, he point tha and t some mame Rutstrom ernatively,the subject addressing such differences in a pool can be de igned to represent a target systematic manner is necessary. oof the ecn e.g.,traders a 2.1 Criteria that Define Field Experiments Board of Trade in Michae Haigh and John List 2004)or the general Running the risk of oversimplifying what population (e.g.,the Danish population in is inherently a multidimensional issue,we Harrison,Morton Igel Lau,and Melonie propose six factors that can be used to deter- Williams 2002). mine the field context of an experiment: In addition,nonstandard subject pools the nature of the subject pool, might bring experience with the commodity the nature of the information that the or the task to the experiment,quite apart subjects bring to the task. from their wider array of demogra hic char the nature of the commodity. acteristics.In the field,subjects bring cer- the nature of the task or trading rules tain information to their trading activities in addition to their knowledge of the trading the nature of the environment that the tance of this information subiect op design and that can lead behavioral at the outset that thes ng abs ch info rrelated to theles of field can be premiu ments that a ta we bon OD] can an mpo ween and field exper tant part of results e common be as the growth Recent years have xperiments concerne ting valuati actual goods standard P00 by rathe than using induced valuations over menters,simply be use the are a conven virtua goods The distinction here is ience sample for academics.Thus when one betwe cen physical goods or actual services ects.the and abstractly defined goods.The latter as nonstandard in this have been the staple of experimental eco sense.But we argue that the use of nonstar nomics since Edward Chamberlin (1948) dard subjects should not automatically qual and Vernon Smith(1962),but imposes an ify the experiment as a field experiment.The artificiality that could be a factor influenc experiments of Cummings, Harrison.and e ing behavior.'Such influences are actually Elizabet Rutstrom(1995),for example,used ot great interest,or should be.If the nature individuals recruited from churches in order of the commodity itself affects behavior ir to obtain a wider range of de a way that is not accounted for by the the characteristics than one standard college setting.The of he I domain of a a nonstandard ool varies from worse is sim eriment to experiment:in this case it sim ply false.In either one can better provideda rated se of s ohic char cte ith el.which ()h nt whe 2 0P1 and s o to nt (9 adjust for D. This content downl
Journal of Economic Literature, Vol. XLII (December 2004) differences? We believe field experiments have matured to the point that some frame- work for addressing such differences in a systematic manner is necessary. 2.1 Criteria that Define Field Experiments Running the risk of oversimplifying what is inherently a multidimensional issue, we propose six factors that can be used to deter- mine the field context of an experiment: * the nature of the subject pool, * the nature of the information that the subjects bring to the task, * the nature of the commodity, * the nature of the task or trading rules applied, * the nature of the stakes, and * the nature of the environment that the subject operates in. We recognize at the outset that these characteristics will often be correlated to varying degrees. Nonetheless, they can be used to propose a taxonomy of field experi- ments that will, we believe, be valuable as comparisons between lab and field experi- mental results become more common. Student subjects can be viewed as the standard subject pool used by experi- menters, simply because they are a conven- ience sample for academics. Thus when one goes "outdoors" and uses field subjects, they should be viewed as nonstandard in this sense. But we argue that the use of nonstan- dard subjects should not automatically qual- ify the experiment as a field experiment. The experiments of Cummings, Harrison, and E. Elizabet Rutstr6m (1995), for example, used individuals recruited from churches in order to obtain a wider range of demographic characteristics than one would obtain in the standard college setting. The importance of a nonstandard subject pool varies from experiment to experiment: in this case it sim- ply provided a less concentrated set of socio- demographic characteristics with respect to age and education level, which turned out to be important when developing statistical models to adjust for hypothetical bias (McKinley Blackburn, Harrison, and Rutstr6m 1994). Alternatively, the subject pool can be designed to represent a target population of the economy (e.g., traders at the Chicago Board of Trade in Michael Haigh and John List 2004) or the general population (e.g., the Danish population in Harrison, Morton Igel Lau, and Melonie Williams 2002). In addition, nonstandard subject pools might bring experience with the commodity or the task to the experiment, quite apart from their wider array of demographic char- acteristics. In the field, subjects bring cer- tain information to their trading activities in addition to their knowledge of the trading institution. In abstract settings the impor- tance of this information is diminished, by design, and that can lead to behavioral changes. For example, absent such informa- tion, risk aversion can lead to subjects requiring a risk premium when bidding for objects with uncertain characteristics. The commodity itself can be an impor- tant part of the field. Recent years have seen a growth of experiments concerned with eliciting valuations over actual goods, rather than using induced valuations over virtual goods. The distinction here is between physical goods or actual services and abstractly defined goods. The latter have been the staple of experimental eco- nomics since Edward Chamberlin (1948) and Vernon Smith (1962), but imposes an artificiality that could be a factor influenc- ing behavior.7 Such influences are actually of great interest, or should be. If the nature of the commodity itself affects behavior in a way that is not accounted for by the the- ory being applied, then the theory has at best a limited domain of applicability that we should be aware of, and at worse is sim- ply false. In either case, one can better 7 It is worth noting that neither Chamberlin (1948) nor Smith (1962) used real payoffs to motivate subjects in their market experiments, although Smith (1962) does explain how that could be done and reports one experiment (fn 9., p. 121) in which monetary payoffs were employed. 1012 This content downloaded from 218.106.182.180 on Sat, 11 Jun 2016 06:18:54 UTC All use subject to http://about.jstor.org/terms
Harrison and List:Field Experiments 1013 nerstand themitations of the geperality this is an important component of the inter- of theory only via empirical testing. play between the lab and field.Early illus Again,however.just having one field char trations of the value of this approach include acteristic,in this case a physical good,does David Grether,R.Mark Isaac,and Charles not constitute a field experiment in any fun Plott [1981,1989],Grether and Plott [1984] damental sense Rutstrom (1998)sold lots and lames Hong and Plott [19821 and lots of chocolate truffles in a laboratory The nature of the stakes can also affect study of different auction institution field responses. Stakes in the laboratory designed to elicit values truthfully,but hers might be very different than those encou was very much a lab e neriment desnite the tered in the field,and hence have an effect tastiness of the Similarly.Iar on behavior if valuations are taken seriousl Bateman et al.(1997)elicited valuations ove when they are in the t s of dollars,or in the nd dessert voucher s for a local res undred made ant While de com wer th c dollar bu nts with stakes beloy the t to obtai a ould e nge em,they are not abs ract.Ther nder imprecise e are many e goods in the o of de mental noditie ation p to differe The natu tas that the subject is periments in relative being asked undertake poor co mes offe e opp ortunity to eva component of a field experiment,since on uate the effects of substantial stakes within a would expect that fiele coul given budget. play a major role in helping individua The environment of the experiment can develop heuristics for specific tasks.The lab also influence behavior.The environmen experiments of John Kagel and Dan Levin can provide context to suggest strategies and (1999)illustrate this point,with"super-expe heuristics that a lab setting might not.Lab rienced"subiects behaving differently thar experimenters have always wondered inexperienced subiects in terms ot thei whether the use of classrooms might engen- propensity to fall prey to the winners'curse der role-playing behavior,and indeed this is one of the reasons experimental economists settings "travel"to the other field and lab onetary rewards.Even settings (Harrison and List 2003).Another with salient rewards.however,environmen t of the task is the st rameteri tal effects could remain.Rather than viev ion that is adopted the xpe controlled effects,we see them as One worthy of controlled study ted f the 2.2 A Proposed Taxonomy ,50 as o bel Mior in a rele is of Any of field domain-specific c,and bel vior can always be ng rentia lab of Chamberlin (1948 There som er,m naving e power of oad terms to differ rentiate wh see a would the key differences.We propose the following terminology a conventional lab experiment is one This content downle a20g2016o61s4UTC
Harrison and List: Field Experiments understand the limitations of the generality of theory only via empirical testing.8 Again, however, just having one field char- acteristic, in this case a physical good, does not constitute a field experiment in any fun- damental sense. Rutstr6m (1998) sold lots and lots of chocolate truffles in a laboratory study of different auction institutions designed to elicit values truthfully, but hers was very much a lab experiment despite the tastiness of the commodity. Similarly, Ian Bateman et al. (1997) elicited valuations over pizza and dessert vouchers for a local restau- rant. While these commodities were not actual pizza or dessert themselves, but vouchers entitling the subject to obtain them, they are not abstract. There are many other examples in the experimental literature of designs involving physical commodities.9 The nature of the task that the subject is being asked to undertake is an important component of a field experiment, since one would expect that field experience could play a major role in helping individuals develop heuristics for specific tasks. The lab experiments of John Kagel and Dan Levin (1999) illustrate this point, with "super-expe- rienced" subjects behaving differently than inexperienced subjects in terms of their propensity to fall prey to the winners' curse. An important question is whether the suc- cessful heuristics that evolve in certain field settings "travel" to the other field and lab settings (Harrison and List 2003). Another aspect of the task is the specific parameteri- zation that is adopted in the experiment. One can conduct a lab experiment with parameter values estimated from the field data, so as to study lab behavior in a "field- relevant" domain. Since theory is often domain-specific, and behavior can always be, 8 To use the example of Chamberlin (1948) again, List (2004e) takes the natural next step by exploring the pre- dictive power of neoclassical theory in decentralized, nat- urally occurring field markets. 9 We would exclude experiments in which the com- modity was a gamble, since very few of those gambles take the form of naturally occurring lotteries. this is an important component of the inter- play between the lab and field. Early illus- trations of the value of this approach include David Grether, R. Mark Isaac, and Charles Plott [1981, 1989], Grether and Plott [1984], and James Hong and Plott [1982]. The nature of the stakes can also affect field responses. Stakes in the laboratory might be very different than those encoun- tered in the field, and hence have an effect on behavior. If valuations are taken seriously when they are in the tens of dollars, or in the hundreds, but are made indifferently when the price is less than one dollar, laboratory or field experiments with stakes below one dol- lar could easily engender imprecise bids. Of course, people buy inexpensive goods in the field as well, but the valuation process they use might be keyed to different stake levels. Alternatively, field experiments in relatively poor countries offer the opportunity to eval- uate the effects of substantial stakes within a given budget. The environment of the experiment can also influence behavior. The environment can provide context to suggest strategies and heuristics that a lab setting might not. Lab experimenters have always wondered whether the use of classrooms might engen- der role-playing behavior, and indeed this is one of the reasons experimental economists are generally suspicious of experiments without salient monetary rewards. Even with salient rewards, however, environmen- tal effects could remain. Rather than view them as uncontrolled effects, we see them as worthy of controlled study. 2.2 A Proposed Taxonomy Any taxonomy of field experiments runs the risk of missing important combinations of the factors that differentiate field experi- ments from conventional lab experiments. There is some value, however, in having broad terms to differentiate what we see as the key differences. We propose the following terminology: * a conventional lab experiment is one 1013 This content downloaded from 218.106.182.180 on Sat, 11 Jun 2016 06:18:54 UTC All use subject to http://about.jstor.org/terms