Harrison and List:Field Experiments 1019 The reason is simple to understand.It is much easier to predict the behavior of a 26- es of students the same as vear-old when one has a model that is based either responses of the behav nonstudent This car be assun or,better yet mate the tested by sampling nonstudents as well as behavior 69-year-old based on the students. ral model fre a sample whose ages Of course,it is always better to be fore- 19to27 casting on the basis of an interpolation relevanc e of these methods for rather than an extrapolation,and that is the the sm of xpe subjects of the ex proce most important problem one has with stu- dent sar oles This issue is discussed in some convenien ce sample in the HLapproach detail by Blackburn.Harrison,and Rutstrom The lessons that are learned from this stu (1994).They estimated a statistical model of dent sample could be embodied in a statisti- subiect cal model of their behavior.with implications stude ts and also a statistica drawn for a larger target population. field Although this approach rests on an assump tion that is as yet untested,concerning the sam area. were conve representativeness of student behavioral samples. T res he onl difference ttohpebss oonses conditional on their characteris ted a much wider tics,it does provide a sir emographic char conclusion s abou acteristics. In the church sa mple. ages method r pop ranged from 21 to 79;in the student sample How cou id this to int ages ranged from 19 to 27.When predicting behavior of students based on the church on which the estimated behavioral model,interpolation ehaviora model show a wa was used and the predictions were extreme portant determinant of avior. nsid ly accurate.in the re rse direction,howev further a situation in whi ch the sample used er,when predicting r church behavior from to estimate the model had an average age tha the student stim dictio oral model,the was not representative of the population as a in the whole.In this case,it is perfectly possible tha ing extremel ariance the responses of the student san For example. population.Although no such instances have eared in the of this pereent men. onsonesdf not be mply a m We clude the e,that ny of the nd,re valid de ra ge of d no o be exte sions of the met ers cu unde rent use. Mor over. extension auxili would general mness of the sam experimental methods obtained with student convenience sa mples pr Further problems arise if one allows unob- a tion research eg.the pro to served individual effects to play a role.In ssible correlation between the two questions). some statistical settings it is possible to allow
1020 Journal of Economic Literature,Vol.XLII(December 2004 for those effects by means of"fixed effect"or beauty.Again,the immediate implication is rand om effects"analyses.But these stan to collect a standard battery of measures of dard devices,now quite common in the tool individual characteristics to allow some sta kit of experimental economists.do no tistical comparisons of conditional treatment address a deeper problem.The internal effects to be drawn.2 But even here we can validity of a randomized design is maximized only easily condition on observable charac- when one knows that the samples in each teristics,and additional identifying assump treatment are identical.This happy extreme tions will be needed to allow for correlated leads many to infer that matching subjects differences in unobservables. on a finite set of characteristics must be bet ter in terms of intemal validity than 4.4 Precursors not characte stics eral experimenters have used artefac But tial ing.The most can be thar tual field that is hey h n (9 and brought n the egel and Heckma 1998 into labs It is notable that this effort has who critique paired-audit tests of discrimi occurred from the earli iest days of exper nation.In these experiments,two applicants mental economics,and that it has only for a job are matched in terms of certain recently become common. observables,such as age,sex,and education Lichtenstein and Slovic (1973)replicated and differ in only one protected characteris their earlier experiments on "preference tic.such as race.However,unless some reversals”in" a nonlaboratory real-pla extremely strong assumptions about how setting unique to the experimental litera. characteristics map into wages are made ture on de a casino in there will be a predetermined bias in out comes.The dire ction of the bias"depends, ofessio nal dealer and the and on t sav 99. ore a ugh the the floor of the from Heckman 110)illus Alth Boys and girl age ar ompetitio height on PDP. highe ak nputer,a D rage boys have a RT.an ard).t was to ide variance in their jumping technique,for any tify gambl rs in the natural habita h number o reasons.I he ject pool of 44 include seven k dealers who worked in Las Ve egas,and the like better jumpers;if the bar is set very dealer's impression was that the game high then the boys will look like bette attracted a higher proportion of profession- jumpers.The implications for numerou al and educated persona than the usual (lab and field)experimental studies of the casino clientele"(p 18) effect of gender.that do not control for Kagel Battalio and lames walker (1979 other characteristics,should be apparent. This metaphor also serves to mind us They were that what laboratory as a "standard im. nters think of artif cts" in lab eed not be e ra m the istic Alth stu o the issue of sample hav roug e Lowenstein (1999)offer a similar criticis na all of the characteristics such as intelligence and zing to treatment from the same population
Harrison and List:Field Experiments 1021 selection bias.28 They conducted a field reason for trade in this onment.30 The ent in the ho of the major empi city de erved price e bubble of the 2 experiments can be said to have had some on usage, energy conservation price bubble informatio They also examined a compar In an effort to address the criticism that ison sample drawn from the same popula bubbles were just a manifestation of using tion,to check for any biases in the volunteer student subjects,Smith, Suchanek.and sample. Williams (1988)recruited nonstudent sub- Binswanger (1980 1981)conducted jects for one experiment.As they put it,one experiments eliciting measures of risk aver- experiment is noteworthy because of its sion from farmers in rural India.Apart from use of professional and business ple from the policy interest of studving agents in one stated goal of the Tucson community.as subie cts.Thi developing countries market belies any notion that our results are using artefactual field experiments was to assess risk attitudes for choices s in which the ne the to have ra real ino the ex ntal task 0 nal substan tial frac alth ethodhe ve conduc T tha clos a mean p than I a ention (p. 130-31.The Charles Hoctin with student sul ects by ference at th to the e observation and Susan Laury(2002) that the price bubble did not burst as the Bums1985)con I induced-value finite norizon of the experiment wa market experiments with floor traders from approaching.Another notable feature of this wool markets.to compare with the behav price bubble is that it was accompanied by ior of student subjects in such settings.The heavy volume,unlike the price bubbles goal was to see if the heuristics and deci. observed with experienced subjects.3 sion rules these traders evolved in their Although these subjects were not students natural field setting affected their behavior ere inexperienced in the use of the She did find that their natural field rivalr double auction experi had a that their field exp rier for this type of asse Arlin n Willia ms(199 con of with e experimen they report nine t twelve trading below ex doub auc traded a number of fifteen ri-vers cte dividend or thirty period assets with the same com 、Th secon mon value distribution of dividends.If all subjects are risk neutral and have common price expectations,then there would be no price ce on bubbles xperienced,or market ts in which th is a sho -lived t only twice experienced in asset market trading. there is some nontrivial volume assoclated with it
1022 Journal of Economic Literature,Vol.XLII (December 2004 Artefactual field experiments have also tasks,experic ce is generated in the field made use of children and high school sub and not the lab.These results provide sup jects.For example,William Harbaugh and port for the notion that context-specific Kate Krause (2000),Harbaugh,Krause,and experience does appear to carry over to com- Timothy Berry (2001),and Harbaugh parable settings,at least with respect to Krause,and Lise Vesterlund(2002)explore these types of auctions other-regarding preferences,individual This experimental design emphasizes the rationality,and risk attitudes among children identification of a naturally occurring setting in school environments. in which one can control for ext Joseph Henrich (2000)and Henrich and the way that it is accumulated in the field ard Meelreath (2002)and Henrich e rienced traders gain al.2001.2004)ha taker field the true wlds” tim by obser nd surviving of range r of some proced cultur thropology pe recruit and instruct subjects in the ab.but artefactu eld experiments. Their focu was on the ulti matum bargaining game and reliably capture the ful measures of risk aversion. extent of the field counterpart of experience. This is not a criticism of lab experiments, just their domain of applicability. 5.Framed Field Experiments The methodological lesson we draw is that 5.1 The Nature of the Information Subjects one should be careful not to generalize from Already Have the evidence of a winner's curse by student subjects that have no experience at all with Auction theory provides a rich set of pre the field context.These results do r not in dictions concerning bidders'behavior.One that field context has expe en ed"ph particularly salient finding in a plethora of es aboratory experiments that is not predicted e such the that win curs they nda enta I need to pre to exper s c be ving general conclusions.It is no the case that abstract,context-free experi lost mo ments provide more general findings if the it regularly.This woul seem to suggest that experience is a suffi cient condition for an individual bidder to expect such context-free experiments to be avoid the winner's curse.Harrison and List unusually tough tests of economic theory (2003)show that this implication is support- since there is no control for the context that ed when one considers a natural setting in subjects might themselves impose on the which it is relatively easy to identify traders abstract experimental task. that are more or less ex erienced at the task The mai result is that if one ants to In their experiments the experience of sub s about the validity of the iects is either tied to the commodity,the val in the field,then one uation task,and the ase of aucti the contex poen field sports or si that lab experiments es are experiments with values).In all nously assigned and defined in an abstract
Harrison and List:Field Experiments 1023 manner,cannot ubiquitously provide reliable rather than abstract commodities is not insights into field behavior.One might be unique to the field,nor does one have to able to modify the lab experimental design to mimic those field contexts more reliably. and the field.Bu that would make for tion of the robust applic field e equence as of ma simple RequteetHn known as the "winner's curs the Tower of or now w game, been extensively studied define an insider as anyone who has bette by cognitive psychologists (e.g.,J.R.Haye information than other market participants. and H.A.Simon 1974)and more recentl If insiders are present in a market,then one by economists (Tanga McDaniel and might expect that the prevailing prices in the Rutstrom 2001)in some fascinating experi- market will reflect their better information ments.The physical form of the game,as This leads to two general questions about found in all serious Montessori classrooms market performance.First,do insiders fall and in Judea Pearl (1984,p.28),is shown in prey to the winner's curse?Second,does the figure 1. presence of insiders mitigate the winner's picture shows the initial state.in curse for the market as a whole? goal is to The peg 1.The oach adopted by Harrison and undertake sta te in the peg 3. natur of the theory an only of m m tim aris endogenously de may eve bigger disk.The obective is to reach en to impose the remaining contr stat e in the le t number ed to implement a clean expe riment.In othe The "trick"to solving the Tower of Hanoi is words,rather than impose all controls exoge to use backwards induction:visualize the nously on a convenience sample of college final,goal state and use the constraints to fig- students.they find a population in the field ure out what the penultimate state must in which one of the factors of interest arises have looked like (viz.,the tiny disk on the top naturally,where it can be identified easily of peg 3 in the goal state would have to be on and then add the necessary controls.To test peg 1 or peg 2 by itself).Then work back their methodological hypotheses,they also from that ultimate state.again res ctin implement a fully co ntrolled laboratory the constraints(viz.the second smallest dis cts drawn from would have to be lation.We discus eg 2 the s nalles below 5.2 The Nature of the Commodity t der for the thrd d b largest disk Many field experim nts involve real,phys one I peg ave to ical com odities and the values that subj be cle ed,so th sma should be on place on them in their daily lives.This is dis top of the second est disk) tinct from the traditional focus in experi Observation of students in Montessor mental economics on experimenter-induced classrooms makes it clear how they(eventu- valuations on an abstract commodity.often ally)solve the puzzle,when confronted with referred to as"tickets"just to emphasize the lack of any field referent that might suggest 2 for a general treatment. Ronald Harstad,and Rutstrom(2004 a valuation.The use of real commodities