1024 Journal of Economic Literature,Vol.XLII(December 2004) Figure 1.The Tower of Hanoi Game straints and move it posed an table cognitive burden state en then physica ally work for some of the experimental backwards along the es o above It might be tem ink of this as just thought experiment in backwards induction s,instead of a real com- The critical point here is that they temporar- modity and its abstract analogue But we ily violate the constraints of the problem in believe that this example does identify an order to solve it"properly." important characteristic of commodities in Contrast this behavior with the laboratory ideal field experiments:the fact that they subjects in McDaniel and Rutstrom(2001) allow subjects to adopt the representation of They were given a computerized version of the commodity and task that best suits their the game and told to try to solve it.However. objective.In other words,the representation the computerized version did not allow them of the commodity by the subject is an inte- to violate the constraints Hence the labora gral part of how the subject solves the task. tory subiects were uable to use the class One simply cannot untangle them,at least room Montessori method,by which the not easily and naturally. student learns the idea of backwards induc This example also illustrates that off. tion by exploring it with physical ref This is not a design flaw Rutstrom (2001)lab e rim nts,but si h i estateo thd ment of indeed be ind ium stat 3 of th su the state e being the s subjec Although y the es as on that make the r ts of le ourage more ndamenta conceptua ng to niteinfernences understanding of the idea of backwards faces to varying degrees. Th cono-UTC
Journal of Economic Literature, Vol. XLII (December 2004) Figure 1. The Tower of Hanoi Game the initial state. They shockingly violate the constraints and move all the disks to the goal state en masse, and then physically work backwards along the lines of the above thought experiment in backwards induction. The critical point here is that they temporar- ily violate the constraints of the problem in order to solve it "properly." Contrast this behavior with the laboratory subjects in McDaniel and Rutstr6m (2001). They were given a computerized version of the game and told to try to solve it. However, the computerized version did not allow them to violate the constraints. Hence the labora- tory subjects were unable to use the class- room Montessori method, by which the student learns the idea of backwards induc- tion by exploring it with physical referents. This is not a design flaw of the McDaniel and Rutstr6m (2001) lab experiments, but simply one factor to keep in mind when evaluating the behavior of their subjects. Without the physical analogue of the final goal state being allowed in the experiment, the subject was forced to visualize that state conceptually, and to likewise imagine conceptually the penultimate states. Although that may encourage more fundamental conceptual understanding of the idea of backwards induction, if attained, it is quite possible that it posed an insurmountable cognitive burden for some of the experimental subjects. It might be tempting to think of this as just two separate tasks, instead of a real com- modity and its abstract analogue. But we believe that this example does identify an important characteristic of commodities in ideal field experiments: the fact that they allow subjects to adopt the representation of the commodity and task that best suits their objective. In other words, the representation of the commodity by the subject is an inte- gral part of how the subject solves the task. One simply cannot untangle them, at least not easily and naturally. This example also illustrates that off- equilibrium states, in which one is not opti- mizing in terms of the original constrained optimization task, may indeed be critical to the attainment of the equilibrium state.33 33 This is quite distinct from the valid point made by Smith (1982, p. 934, fn. 17), that it is appropriate to design the experimental institution so as to make the task as sim- ple and transparent as possible, providing one holds con- stant these design features as one compares experimental treatments. Such designs may make the results of less interest for those wanting to make field inferences, but that is a trade-off that every theorist and experimenter faces to varying degrees. 1024 This content downloaded from 218.106.182.180 on Sat, 11 Jun 2016 06:18:54 UTC All use subject to http://about.jstor.org/terms
Harrison and List:Field Experiments 1025 Thus we should be mindful of possible field reasons why homegrown values might be devices that allow subjects to explore off- affiliated in such experiments equilibrium states,even if those states are The first is that the good being auctioned ruled out in our null hypotheses. might have some uncertain attributes,and fel- Field Goods Have Field Substitutes.There low bidders might have more or less informa are two re pects in which"field substitutes' tion about those ttributes.Depending on hov one p erceives the knowed of other bidders rring, ation of their bidding behavior3 ods.We can refer to the for affect a gi en bidder of the true sub ntext of substitutes,and to the to the extent that the of substit bidder's the lot attribute The e to be elicited,the ed if reliable being au estimate Note that hat is being minimized latter nee The :way in w utes play a crey auction ere (for in an t is the traditional demand th atte ory:to ottle of scotch may sub 5t e for a bible on differ.they when seek ng peace of mind. e degre the substitutability here is the stuff of individual demand elasticities,and can reasonably be expected to vary from subject to subject.The upshot of this consideration is.vet again.that nother one should always collect information on odity b ething w observable individual characteristics and control for them ()m The second way in which substitutes play a role in an exne iment is the more subtle f th issue of affiliation which arises in lab or field not ped a t settin os that involve preference over a field e just o choc ep eated Vickrev ana6otp uctions in which subie ailing prices.This 10 of oont ol sin dealing with elicitation of ho .gre In the wil an ten that ho privat the grown values this sub ar expec ms for the ct on elicited valu sing repeat Vickrey auction. Vickrey auction 82 and it is d than a one-sho the the s Inere are,in turn,two of the ho M The theoretical and ey that the ycomparing real-tim and Ka gcl Harstad.and Le the ce is aucti iect to d ko r than in the vick o。 ion in sach round. were r to bid higher than in the Vickrey auction. This content downle im2806182180 on Sat.1m2016061s4UTc use subject to ht
Harrison and List: Field Experiments Thus we should be mindful of possible field devices that allow subjects to explore off- equilibrium states, even if those states are ruled out in our null hypotheses. Field Goods Have Field Substitutes. There are two respects in which "field substitutes" play a role whenever one is conducting an experiment with naturally occurring, or field, goods. We can refer to the former as the natural context of substitutes, and to the latter as an artificial context of substitutes. The former needs to be captured if reliable valuations are to be elicited; the latter needs to be minimized or controlled. The first way in which substitutes play a role in an experiment is the traditional sense of demand theory: to some individuals, a bottle of scotch may substitute for a bible when seeking peace of mind. The degree of substitutability here is the stuff of individual demand elasticities, and can reasonably be expected to vary from subject to subject. The upshot of this consideration is, yet again, that one should always collect information on observable individual characteristics and control for them. The second way in which substitutes play a role in an experiment is the more subtle issue of affiliation which arises in lab or field settings that involve preferences over a field good. To see this point, consider the use of repeated Vickrey auctions in which subjects learn about prevailing prices. This results in a loss of control, since we are dealing with the elicitation of homegrown values rather than experimenter-induced private values. To the extent that homegrown values are affiliated across subjects, we can expect an effect on elicited values from using repeated Vickrey auctions rather than a one-shot Vickrey auction.34 There are, in turn, two 34 The theoretical and experimental literature makes this point clearly by comparing real-time English auctions with sealed-bid Vickrey auctions: see Paul Milgrom and Robert Weber (1982) and Kagel, Harstad, and Levin (1987). The same logic that applies for a one-shot English auction applies for a repeated Vickrey auction, even if the specific bidding opponents were randomly drawn from the population in each round. reasons why homegrown values might be affiliated in such experiments. The first is that the good being auctioned might have some uncertain attributes, and fel- low bidders might have more or less informa- tion about those attributes. Depending on how one perceives the knowledge of other bidders, observation of their bidding behavior35 can affect a given bidder's estimate of the true sub- jective value to the extent that they change the bidder's estimate of the lottery of attributes being auctioned.36 Note that what is being 35 The term "bidding behavior" is used to allow for information about bids as well as non-bids. In the repeat- ed Vickrey auction it is the former that is provided (for winners in previous periods). In the one-shot English auc- tion it is the latter (for those who have not yet caved in at the prevailing price). Although the inferential steps in using these two types of information differ, they are each informative in the same sense. Hence any remarks about the dangers of using repeated Vickrey auctions apply equally to the use of English auctions. 36 To see this point, assume that a one-shot Vickrey auc- tion was being used in one experiment and a one-shot English auction in another experiment. Large samples of subjects are randomly assigned to each institution, and the commodity differs. Let the commodity be something whose quality is uncertain; an example used by Cummings, Harrison and Rutstrim (1995) and Rutstrim (1998) might be a box of gourmet chocolate truffles. Amongst undergraduate students in South Carolina, these boxes present something of a taste challenge. The box is not large in relation to those found in more common chocolate products, and many of the students have not developed a taste for gourmet chocolates. A subject endowed with a diverse pallet is faced with an uncertain lottery. If these are just ordinary chocolates dressed up in a small box, then the true value to the subject is small (say, $2). If they are indeed gourmet chocolates then the true value to the subject is much higher (say, $10). Assuming an equal chance of either state of chocolate, the risk-neutral sub- ject would bid their true expected value (in this example, $6). In the Vickrey auction this subject will have an incentive to write down her reservation price for this lottery as described above. In the English auction, however, this subject is able to see a number of other subjects indicate that they are willing to pay reasonably high sums for the commodity. Some have not dropped out of the auction as the price has gone above $2, and it is closing on $6. What should the subject do? The answer depends critically on how knowledgeable he thinks the other bidders are as to the quality of the chocolates. If those who have dropped out are the more knowledgeable ones, then the correct inference is that the lottery is more heavily weighted towards these being common chocolates. If those remaining in the auction are the more knowledgeable ones, however, then the opposite inference is appropriate. In the former case the real-time observation should lead the subject to bid lower than in the Vickrey auction, and in the latter case the real-time observation should lead the subject to bid higher than in the Vickrey auction. 1025 This content downloaded from 218.106.182.180 on Sat, 11 Jun 2016 06:18:54 UTC All use subject to http://about.jstor.org/terms