10THEORETICAL ECOLOGYAfter many generations, however, GTFT istime. Most surprisingly,this strategy is based onundermined byunconditional cooperators,ALLC.the extremely simple principle of win-stay, lose-shift (WSLS). If my payoff is R or T then I willIn a society where everybody is nice (usingGTFT),there is almost no need to remember how tocontinuewiththesamemovenextround.IfI haveretaliate against a defection. A biological trait thatcooperated then I will cooperate again, if I haveis not used is likely to be lost by random drift.defected then I will defect again. If my payoff isBirdsthat escapetoislands without predators loseonlySorPthenIwill switchtotheothermovethe ability to fly. Similarly, a GTFT population isnext round. If I have cooperated then I will defect,softened and turns intoan ALLCpopulation.if I have defected thenI will cooperate (Figure2.2)OncemostpeopleplayALLC,thereisanopenIf two WSLS strategists play each other,theyinvitation for ALLD to seize power.This is pre-cooperate most of the time.If a defection occurscisely what happens.The evolutionary dynamicsaccidentally,thenin thenext move bothwillrunincvcles:fromALLDtoTFTtoGTFTtoALLCdefect.Hereafter both will cooperate again.WSLSand backtoALLD.These oscillations of coopera-is a simple deterministic machineto correct sto-tiveand defective societies are a fundamental partchastic noise. While TFT cannot correct mistakes,both GTFTandWSLScan.ButWSLShasanof all our observations regardingthe evolution ofcooperation.Most models of cooperation showadditionalaceinitshand.WhenWSLSplavsALLC it will discoverafter some time that ALLCsuchoscillations.Cooperation is nevera final stateof evolutionarydynamics.Instead it isalways lostdoes not retaliate. After an accidental defection,todefection after sometime andhastobeWSLS will switch to permanent defection.There-re-established.These oscillations are also reminis-fore,apopulationofWSLSplayersdoesnotdrifttocent of alternating episodes of war and peace inALLC.Cooperationbased on WSLSismore stablehuman history (Figure 2.1).than cooperation based on TFT-like strategies.A subsequent set of simulations, exploring alarger strategy space, led to a surprise (Nowak andWin-staySigmund,1993).The fundamental oscillations wereC(3).... CD (5).... Dinterrupted byanother strategy which seems tobeccable to hold its ground for a very long period ofLose-shiftTit-for-tatGenerous tit-for-tatC (0).... DD (1)....C (probabilistic)金具DDFigure 2.2 Win-stay, lose-shift (WSLS) embodies a very simpleAlways defectAlways cooperateprinciple. If you do well then continue with what you are doing. If youare not doing well,thentrysomething else.Here we considerthePrisoner's Dilemma payoff values R=3, T=5, P=1,and S=0. Ifboth players cooperate, you receive three points, and you continue toWin-stay, lose-shiftcooperate. If you defect against a cooperator, you receive five points,and you continue to defect. But if you cooperate with a defector, youFigure 2.1 Evolutionary cycles of cooperation and defection. Areceivenopoints.andthereforevouwillswitchfromcooperationtosmall cluster of tit-for-tat (TFT) players or even a lineage starting fromdefection. If, on the other hand, you defect against a defector, youa single TFT player in a finite population can invade an always defectreceive one point,and you will switch to cooperation. Your aspiration(ALLD) population. In fact, TFT is the most efficient catalyst for thelevel is three points. if you get at least three points then you considerfirstemergenceofcooperationinanALLDpopulation.But inaworldit a win and you will stay with your current choice. If you get lessof fuzzy minds and trembling hands, TFT is soon replaced by generousthan three points, you consider it a loss and you will shift to anothertit-for-tat (GTFT), which can re-establish cooperation after occasionalmove. If R> (T+ P)/2 (or blc> 2) then WSLS is stable againstmistakes. If everybody uses GTFT, then always cooperate (ALLC) is ainvasion by ALLD. If this inequality does not hold, then our evolu-neutralvariant.RandomdriftleadstoALLC.AnALLCpopulationtionary simulations lead to a stochastic variant of WSLS, whichinvites invasion by ALLD.But ALLC is also dominated by win-stay.cooperates after a DD move only with a certain probability. Thislose-shift (WSLS), which leads to more stable cooperation than TFT-stochastic variant of WSLS is then stable against invasion by ALLDlike strategies
After many generations, however, GTFT is undermined by unconditional cooperators, ALLC. In a society where everybody is nice (using GTFT), there is almost no need to remember how to retaliate against a defection. A biological trait that is not used is likely to be lost by random drift. Birds that escape to islands without predators lose the ability to fly. Similarly, a GTFT population is softened and turns into an ALLC population. Once most people play ALLC, there is an open invitation for ALLD to seize power. This is precisely what happens. The evolutionary dynamics run in cycles: from ALLD to TFT to GTFT to ALLC and back to ALLD. These oscillations of cooperative and defective societies are a fundamental part of all our observations regarding the evolution of cooperation. Most models of cooperation show such oscillations. Cooperation is never a final state of evolutionary dynamics. Instead it is always lost to defection after some time and has to be re-established. These oscillations are also reminiscent of alternating episodes of war and peace in human history (Figure 2.1). A subsequent set of simulations, exploring a larger strategy space, led to a surprise (Nowak and Sigmund, 1993). The fundamental oscillations were interrupted by another strategy which seems to be able to hold its ground for a very long period of time. Most surprisingly, this strategy is based on the extremely simple principle of win-stay, loseshift (WSLS). If my payoff is R or T then I will continue with the same move next round. If I have cooperated then I will cooperate again, if I have defected then I will defect again. If my payoff is only S or P then I will switch to the other move next round. If I have cooperated then I will defect, if I have defected then I will cooperate (Figure 2.2). If two WSLS strategists play each other, they cooperate most of the time. If a defection occurs accidentally, then in the next move both will defect. Hereafter both will cooperate again. WSLS is a simple deterministic machine to correct stochastic noise. While TFT cannot correct mistakes, both GTFT and WSLS can. But WSLS has an additional ace in its hand. When WSLS plays ALLC it will discover after some time that ALLC does not retaliate. After an accidental defection, WSLS will switch to permanent defection. Therefore, a population of WSLS players does not drift to ALLC. Cooperation based on WSLS is more stable than cooperation based on TFT-like strategies. Tit-for-tat Generous tit-for-tat Always defect Always cooperate Win-stay, lose-shift Figure 2.1 Evolutionary cycles of cooperation and defection. A small cluster of tit-for-tat (TFT) players or even a lineage starting from a single TFT player in a finite population can invade an always defect (ALLD) population. In fact, TFT is the most efficient catalyst for the first emergence of cooperation in an ALLD population. But in a world of fuzzy minds and trembling hands, TFT is soon replaced by generous tit-for-tat (GTFT), which can re-establish cooperation after occasional mistakes. If everybody uses GTFT, then always cooperate (ALLC) is a neutral variant. Random drift leads to ALLC. An ALLC population invites invasion by ALLD. But ALLC is also dominated by win-stay, lose-shift (WSLS), which leads to more stable cooperation than TFTlike strategies. C C D D Lose-shift C (0) . D D (1) . C (probabilistic) Win-stay C (3) . C D (5) . D Figure 2.2 Win-stay, lose-shift (WSLS) embodies a very simple principle. If you do well then continue with what you are doing. If you are not doing well, then try something else. Here we consider the Prisoner’s Dilemma payoff values R ¼ 3, T ¼ 5, P ¼ 1, and S ¼ 0. If both players cooperate, you receive three points, and you continue to cooperate. If you defect against a cooperator, you receive five points, and you continue to defect. But if you cooperate with a defector, you receive no points, and therefore you will switch from cooperation to defection. If, on the other hand, you defect against a defector, you receive one point, and you will switch to cooperation. Your aspiration level is three points. If you get at least three points then you consider it a win and you will stay with your current choice. If you get less than three points, you consider it a loss and you will shift to another move. If R > (T þ P)/2 (or b/c > 2) then WSLS is stable against invasion by ALLD. If this inequality does not hold, then our evolutionary simulations lead to a stochastic variant of WSLS, which cooperates after a DD move only with a certain probability. This stochastic variant of WSLS is then stable against invasion by ALLD. 10 THEORETICAL ECOLOGY
11HOWPOPULATIONSCOHERETherepeatedPrisoner'sDilemmaismostlyknowndisplay a large amount of cooperation betweenas a story of TFT, but WSLS is a superior strategynon-relatives (Fehr and Fischbacher,2003). A con-in an evolutionary scenario with errors, mutation,siderable part of human cooperation is based onand many generations (Fudenberg and Maskin,moralistic emotions, such as anger directed towards1990; Nowak and Sigmund, 1993).cheaters or the warm inner glowfelt after perform-In the infinitely repeated game, WSLS is stableing an altruistic action.Intriguingly,humans notagainst invasion by ALLD if b/c>2.If insteadonly feel strongly about interactions that involve1<b/c<2thena stochastic variant of WSLSthem directly,they also judge actions between thirddominates the scene; this strategy cooperates afterparties as evidenced by the contents of gossip.Therea mutual defection only with a certain probabilityare numerous experimental studies of indirectOf course,all strategies of direct reciprocity,suchreciprocity based on reputation (Wedekind andasTFT,GTFT,orWSLS canonlylead totheevo-Milinski, 2000;Milinski et al.,2002;Wedekind andlution of cooperation if the fundamental inequalityBraithwaite,2002; Seinen and Schram,2006)(eqn 2.2) is fulfilled.Asimplemodelof indirectreciprocity(NowakandSigmund,1998a,1998b)assumesthatwithinawell-mixedpopulation,individuals meetran-2.3 Indirect reciprocitydomly, one in the role of the potential donor, theWhereas direct reciprocity embodies the idea ofother as potential recipient. Each individualyou scratchmybackandIscratchyours,indirectexperiences several rounds of this interaction inreciprocity suggests that you scratch my back andboth roles, but never with the same partner twice.I scratch someone else's. Why should this work?Aplayer can follow either an unconditional strat-Presumably I will not get scratched if it becomesegy,such as always cooperate or always defect, orknown that I scratch nobody. Indirect reciprocity,a conditional strategy,which discriminatesamonginthisview,is based on reputation(Nowakandthepotential recipients according to their pastSigmund,1998a,1998b,2005).But why should youinteractions.Inasimpleexample,a discriminatingcare about what I do to a third person?donor helps a recipient if her score exceeds aThe main reason why economists and socialcertain threshold. A player's score is O at birth,scientists are interested in indirect reciprocity isincreases whenever that player helps and decrea-because one-shotinteractions betweenanonymousses whenever the player withholds help. Indivi-partners in a global market become increasinglydual-based simulations and direct calculationsshow that cooperation based on indirect reci-frequent and tend to replace the traditional long-lasting associations and long-term interactionsprocity can evolve provided the probability,q,ofbetween relatives,neighbors,or members of theknowingthesocialscoreofanotherpersonexceedssame village. Again, as for kin selection, it is athe cost/benefit ratio of the altruistic act:question of the size of the group.A substantial part(2.3)q>c/bof our life is spent in the company of strangers,and many transactions are no longer face to face.The role of genetic relatedness that is crucial forThe growthof online auctions and other forms ofkin selection is replaced by social acquaintance-e-commerce is based,toaconsiderabledegree,onship. In a fluid population, where most inter-reputation and trust.Thepossibility to exploit suchtrust raises what economists call moral hazards.actions are anonvmousandpeoplehavenopossibility of monitoring the social score of others,Howeffectiveisreputation,especiallyif informa-indirect reciprocity has no chance. But in a sociallytion is only partial?viscous population,wherepeopleknow each other'sEvolutionary biologists, on the other hand, arereputation, cooperation by indirect reciprocityinterested in the emergence of human societies,can thrive (Nowak and Sigmund, 1998a).whichconstitutesthelast (up tonow)of themajorIn a world of binary moral judgements (Nowaktransitions in evolution.In contrast to other eusocialand Sigmund, 1998b; Leimar and Hammerstein,species, such as bees, ants, or termites,humans
The repeated Prisoner’s Dilemma is mostly known as a story of TFT, but WSLS is a superior strategy in an evolutionary scenario with errors, mutation, and many generations (Fudenberg and Maskin, 1990; Nowak and Sigmund, 1993). In the infinitely repeated game, WSLS is stable against invasion by ALLD if b/c > 2. If instead 1 < b/c < 2 then a stochastic variant of WSLS dominates the scene; this strategy cooperates after a mutual defection only with a certain probability. Of course, all strategies of direct reciprocity, such as TFT, GTFT, or WSLS can only lead to the evolution of cooperation if the fundamental inequality (eqn 2.2) is fulfilled. 2.3 Indirect reciprocity Whereas direct reciprocity embodies the idea of you scratch my back and I scratch yours, indirect reciprocity suggests that you scratch my back and I scratch someone else’s. Why should this work? Presumably I will not get scratched if it becomes known that I scratch nobody. Indirect reciprocity, in this view, is based on reputation (Nowak and Sigmund, 1998a, 1998b, 2005). But why should you care about what I do to a third person? The main reason why economists and social scientists are interested in indirect reciprocity is because one-shot interactions between anonymous partners in a global market become increasingly frequent and tend to replace the traditional longlasting associations and long-term interactions between relatives, neighbors, or members of the same village. Again, as for kin selection, it is a question of the size of the group. A substantial part of our life is spent in the company of strangers, and many transactions are no longer face to face. The growth of online auctions and other forms of e-commerce is based, to a considerable degree, on reputation and trust. The possibility to exploit such trust raises what economists call moral hazards. How effective is reputation, especially if information is only partial? Evolutionary biologists, on the other hand, are interested in the emergence of human societies, which constitutes the last (up to now) of the major transitions in evolution. In contrast to other eusocial species, such as bees, ants, or termites, humans display a large amount of cooperation between non-relatives (Fehr and Fischbacher, 2003). A considerable part of human cooperation is based on moralistic emotions, such as anger directed towards cheaters or the warm inner glow felt after performing an altruistic action. Intriguingly, humans not only feel strongly about interactions that involve them directly, they also judge actions between third parties as evidenced by the contents of gossip. There are numerous experimental studies of indirect reciprocity based on reputation (Wedekind and Milinski, 2000; Milinski et al., 2002; Wedekind and Braithwaite, 2002; Seinen and Schram, 2006). A simple model of indirect reciprocity (Nowak and Sigmund, 1998a, 1998b) assumes that within a well-mixed population, individuals meet randomly, one in the role of the potential donor, the other as potential recipient. Each individual experiences several rounds of this interaction in both roles, but never with the same partner twice. A player can follow either an unconditional strategy, such as always cooperate or always defect, or a conditional strategy, which discriminates among the potential recipients according to their past interactions. In a simple example, a discriminating donor helps a recipient if her score exceeds a certain threshold. A player’s score is 0 at birth, increases whenever that player helps and decreases whenever the player withholds help. Individual-based simulations and direct calculations show that cooperation based on indirect reciprocity can evolve provided the probability, q, of knowing the social score of another person exceeds the cost/benefit ratio of the altruistic act: q > c=b ð2:3Þ The role of genetic relatedness that is crucial for kin selection is replaced by social acquaintanceship. In a fluid population, where most interactions are anonymous and people have no possibility of monitoring the social score of others, indirect reciprocity has no chance. But in a socially viscous population, where people know each other’s reputation, cooperation by indirect reciprocity can thrive (Nowak and Sigmund, 1998a). In a world of binary moral judgements (Nowak and Sigmund, 1998b; Leimar and Hammerstein, HOW POPULATIONS COHERE 11
12THEORETICALECOLOGY2001;Fishman, 2003;Panchanathan and Boyd,defection, D,always leads to a bad reputation, B.2003;BrandtandSigmund,2004,2005),thereareStanding (Sugden, 1986) is like scoring, but it is notbad if a good donor defects against a bad recipient.four ways of assessing donors in terms of first-order assessment: always consider them as good,With judging,in addition, it is bad to cooperatealways considerthem as bad, consider them aswith a bad recipient.For another assessment rule,good if they refuse to give,or consider them asshunning, all donors who meet a bad recipientgood if they give. Only this last option makesbecome bad, regardless of what action they choose.Shunning strikes us as grossly unfair, but itsense.Second-order assessment also depends onthe score of the receiver; for example,it can beemerges as the winner in a computertournamentdeemed good to refuse help to a bad person. Thereif errors in perception are included and if there areare16second-orderrules.Third-orderassessmentonly a few rounds in the game (Takahashi andalsodependson thescoreof thedonor:forMashima,2003)example,agoodpersonrefusingtohelpabadAn action rule for indirect reciprocity prescribesperson may remain good, but a bad person refus-giving or not giving, depending on the scores ofing to help a bad person remains bad.There areboth donor and recipient.For example,you may256third-orderassessmentrules.Wedisplayfourdecide to help if the recipient's score isgood orof them in Figure 2.3.your own score is bad.Such an action mightWith the scoring assessment rule, cooperation,increase your own score and therefore increaseC, always leads to a good reputation, G, whereasthe chance of receiving help in the future. Thereare 16action rules.If we view a strategy as the combination of anReputation of donor and recipientactionruleandanassessmentrule,weobtain4096GGGBBGBBstrategies.In a remarkable calculation, OhtsukiGGGCGandIwasa(2004,2005)analyzedall4096strategiesScoringand proved that only eight of them are evolutio-BBBDBnarily stable under certain conditions and lead tocooperation (Figure2.4).UGGGGooooStandingBoth standing and judging belong to the leadingDbGBBeight, but scoring and shunning are not. However,we expect that scoring has a similar role in indirectBcGGBreciprocity to that of TFT in direct reciprocity.JudgingDBGBBNeither strategy is evolutionarily stable, but theirsimplicity and their ability to catalyze cooperationChGGBin adverse situations constitute their strength. InShunningBBBDBextended versions of indirect reciprocity,in whichdonors can sometimes deceive others about theReputation of donorreputation of the recipient,scoring is the foolproofaftertheactionconcept of T believe what I see'. Scoring judgesFigure 2.3 Four assessment rules. Assessment rules specify how anthe action and ignores the stories. There is alsoobserverjudgesaninteractionbetweenapotentialdonorandaexperimental evidencethathumans follow scoringrecipient. Here we show four examples of assessment rules in a worldrather than standing (Milinski et al., 2001).of binary reputation, good (G) and bad (B). For scoring, cooperationIn human evolution,there must have been a(C) earns a good reputation and defection (D) earns a bad reputation.tendencyto move from the simple cooperationStanding is very similar to scoring: the only difference is that a gooddonor can defect aqainst a bad recipient without losing his qoodpromoted bykin or group selection to the strategicreputation. Note that scoring is associated with costly punishmentsubtleties of direct and indirect reciprocity.Direct(Siqmundetal.,2001;:FehrandGaechter,2002),whereasforreciprocityrequires precise recognition of indivistanding punishment of bad recipients is cost-free. For judging it isdual people,amemory of thevarious interactionsbad to help a bad recipient. Shunning assigns a bad reputation to anyone had with them in the past, and enough braindonor who interacts with a bad recipient
2001; Fishman, 2003; Panchanathan and Boyd, 2003; Brandt and Sigmund, 2004, 2005), there are four ways of assessing donors in terms of firstorder assessment: always consider them as good, always consider them as bad, consider them as good if they refuse to give, or consider them as good if they give. Only this last option makes sense. Second-order assessment also depends on the score of the receiver; for example, it can be deemed good to refuse help to a bad person. There are 16 second-order rules. Third-order assessment also depends on the score of the donor; for example, a good person refusing to help a bad person may remain good, but a bad person refusing to help a bad person remains bad. There are 256 third-order assessment rules. We display four of them in Figure 2.3. With the scoring assessment rule, cooperation, C, always leads to a good reputation, G, whereas defection, D, always leads to a bad reputation, B. Standing (Sugden, 1986) is like scoring, but it is not bad if a good donor defects against a bad recipient. With judging, in addition, it is bad to cooperate with a bad recipient. For another assessment rule, shunning, all donors who meet a bad recipient become bad, regardless of what action they choose. Shunning strikes us as grossly unfair, but it emerges as the winner in a computer tournament if errors in perception are included and if there are only a few rounds in the game (Takahashi and Mashima, 2003). An action rule for indirect reciprocity prescribes giving or not giving, depending on the scores of both donor and recipient. For example, you may decide to help if the recipient’s score is good or your own score is bad. Such an action might increase your own score and therefore increase the chance of receiving help in the future. There are 16 action rules. If we view a strategy as the combination of an action rule and an assessment rule, we obtain 4096 strategies. In a remarkable calculation, Ohtsuki and Iwasa (2004, 2005) analyzed all 4096 strategies and proved that only eight of them are evolutionarily stable under certain conditions and lead to cooperation (Figure 2.4). Both standing and judging belong to the leading eight, but scoring and shunning are not. However, we expect that scoring has a similar role in indirect reciprocity to that of TFT in direct reciprocity. Neither strategy is evolutionarily stable, but their simplicity and their ability to catalyze cooperation in adverse situations constitute their strength. In extended versions of indirect reciprocity, in which donors can sometimes deceive others about the reputation of the recipient, scoring is the foolproof concept of ‘I believe what I see’. Scoring judges the action and ignores the stories. There is also experimental evidence that humans follow scoring rather than standing (Milinski et al., 2001). In human evolution, there must have been a tendency to move from the simple cooperation promoted by kin or group selection to the strategic subtleties of direct and indirect reciprocity. Direct reciprocity requires precise recognition of individual people, a memory of the various interactions one had with them in the past, and enough brain Reputation of donor and recipient Reputation of donor after the action Scoring Standing Judging Shunning Action of donor GG CG G G G DB B B B CG G G G DB G B B CG B G B DB G B B CG B G B DB B B B GB BG BB Figure 2.3 Four assessment rules. Assessment rules specify how an observer judges an interaction between a potential donor and a recipient. Here we show four examples of assessment rules in a world of binary reputation, good (G) and bad (B). For scoring, cooperation (C) earns a good reputation and defection (D) earns a bad reputation. Standing is very similar to scoring; the only difference is that a good donor can defect against a bad recipient without losing his good reputation. Note that scoring is associated with costly punishment (Sigmund et al., 2001; Fehr and Gaechter, 2002), whereas for standing punishment of bad recipients is cost-free. For judging it is bad to help a bad recipient. Shunning assigns a bad reputation to any donor who interacts with a bad recipient. 12 THEORETICAL ECOLOGY
HOWPOPULATIONSCOHERE13GBBGBBGGOGGAssessmentBBGDCDcC/DAction2h-2c↑If a good donor meets a bad recipient,2b-3cthe donor must defect, and this action doesFigure 2.5 Games on graphs. The members of a population occupynot reduce his reputation.the vertices of a graph (or social network). The edges denote whointeracts with whom. Here we consider the specific example ofcanbesetasGorBGcooperators, C, competing with defectors, D. A cooperator pays aBIf a column in theassessment module iscost, c, for every link. Each neighbor of a cooperator receives athen the actionmust be C, otherwiseD.benefit, b. The payoffs of some individuals are indicated in the fiqureThe fitness of each individual is a constant, denoting the baselinefitness, plus the payoff of the game. For evolutionary dynamics, weFigure 2.4 Ohtsuki and iwasa's leading eight. Ohtsuki and Iwasaassume that in each round a random player is chosen to die, and the(2004, 2005) have analyzed the combination of 2°=256 assessmentneighbors compete for the empty site proportional to their fitness. Amodules with 2°=16 action modules. This is a total of 4096simple rule emerges: if blc > k then selection favors cooperators overstrategies.They have found that eight of these strategies can bedefectors. Here k is the average number of neighbors per individual.evolutionarilystableandleadtocooperation,providedthateverybodyagrees on each other's reputation. (In general, uncertainty andincomplete information might lead to private lists of the reputation ofprovided the very selective scenario that led toothers.) The three asterisks in the assessment module indicate a freechoice between G and B. There are therefore 23=8 differentcerebral expansion in human evolution.assessment rules which make up the leading eight. The action moduleis built as follows: if the column in the assessment module is G and B2.4Graph selectionthen the correspondingaction is C,otherwise the action is D.Notethat standing and judging aremembers of the leading eight, but thatThe traditional model of evolutionary gamescoring and shunning are not.dynamicsassumesthatpopulationsarewell-mixed(Taylor and Jonker,1978;Hofbauerand Sigmund,1998).This means that interactions between anypower to conduct multiple repeatedtwo players are equally likely.More realistically,gamessimultaneously.Indirect reciprocity,in addition,however,the interactionsbetween individuals arerequires the individual to monitor interactionsgoverned by spatial effects or social networks. Letus therefore assume that the individuals of aamongotherpeople,possibly judgetheintentionspopulation occupy thevertices of a graph (Nowakthat occur in such interactions, and keep up withand May,1992;Nakamaru et al.,1997,1998;Skyrmsthe ever-changing social network of the group.andPemantle,2000;Abramson andKuperman,Reputation of players may not only be determined2001;Ebel and Bornholdt,2002;Lieberman et al.,by their own actions, but also by their associations2005;Nakamaru and Iwasa,2005; Santos et al.,2005;with others.Weexpectthat indirect reciprocityhasSantos and Pacheco, 2005).The edges of the graphcoevolved with human language.On the one hand,determine who interacts with whom (Figure 2.5)Consider a population of N individuals consist-it is helpful to have names for other people and toingof cooperators and defectors.A cooperatorreceive information abouthow a person isper-helpsall individualstowhomit is connected,andceived by others.On the other hand a complexlanguage is needed, especially if there are intricatepays a cost, c.If a cooperator is connected to kother individuals and i of those are cooperators,social interactions.Thepossibilities for games ofthen itspayoff isbi-ck.A defector does notpromanipulation,deceit,cooperation,and defectionare limitless. It is likely that indirect reciprocity hasvide any help,and thereforehas no costs,but it
power to conduct multiple repeated games simultaneously. Indirect reciprocity, in addition, requires the individual to monitor interactions among other people, possibly judge the intentions that occur in such interactions, and keep up with the ever-changing social network of the group. Reputation of players may not only be determined by their own actions, but also by their associations with others. We expect that indirect reciprocity has coevolved with human language. On the one hand, it is helpful to have names for other people and to receive information about how a person is perceived by others. On the other hand a complex language is needed, especially if there are intricate social interactions. The possibilities for games of manipulation, deceit, cooperation, and defection are limitless. It is likely that indirect reciprocity has provided the very selective scenario that led to cerebral expansion in human evolution. 2.4 Graph selection The traditional model of evolutionary game dynamics assumes that populations are well-mixed (Taylor and Jonker, 1978; Hofbauer and Sigmund, 1998). This means that interactions between any two players are equally likely. More realistically, however, the interactions between individuals are governed by spatial effects or social networks. Let us therefore assume that the individuals of a population occupy the vertices of a graph (Nowak and May, 1992; Nakamaru et al., 1997, 1998; Skyrms and Pemantle, 2000; Abramson and Kuperman, 2001; Ebel and Bornholdt, 2002; Lieberman et al., 2005; Nakamaru and Iwasa, 2005; Santos et al., 2005; Santos and Pacheco, 2005). The edges of the graph determine who interacts with whom (Figure 2.5). Consider a population of N individuals consisting of cooperators and defectors. A cooperator helps all individuals to whom it is connected, and pays a cost, c. If a cooperator is connected to k other individuals and i of those are cooperators, then its payoff is bi ck. A defector does not provide any help, and therefore has no costs, but it GG GB BG BB C D G B C D C C/D G B * * * G Assessment Action If a good donor meets a bad recipient, the donor must defect, and this action does not reduce his reputation. * can be set as G or B. If a column in the assessment module is then the action must be C, otherwise D. G B Figure 2.4 Ohtsuki and Iwasa’s leading eight. Ohtsuki and Iwasa (2004, 2005) have analyzed the combination of 28 ¼ 256 assessment modules with 24 ¼ 16 action modules. This is a total of 4096 strategies. They have found that eight of these strategies can be evolutionarily stable and lead to cooperation, provided that everybody agrees on each other’s reputation. (In general, uncertainty and incomplete information might lead to private lists of the reputation of others.) The three asterisks in the assessment module indicate a free choice between G and B. There are therefore 23 ¼ 8 different assessment rules which make up the leading eight. The action module is built as follows: if the column in the assessment module is G and B, then the corresponding action is C, otherwise the action is D. Note that standing and judging are members of the leading eight, but that scoring and shunning are not. C C C C C D D D D D 2b – 5c 2b – 2c 2b – 3c b b b Figure 2.5 Games on graphs. The members of a population occupy the vertices of a graph (or social network). The edges denote who interacts with whom. Here we consider the specific example of cooperators, C, competing with defectors, D. A cooperator pays a cost, c, for every link. Each neighbor of a cooperator receives a benefit, b. The payoffs of some individuals are indicated in the figure. The fitness of each individual is a constant, denoting the baseline fitness, plus the payoff of the game. For evolutionary dynamics, we assume that in each round a random player is chosen to die, and the neighbors compete for the empty site proportional to their fitness. A simple rule emerges: if b/c > k then selection favors cooperators over defectors. Here k is the average number of neighbors per individual. HOW POPULATIONS COHERE 13
14THEORETICALECOLOGYcan receive the benefit from neighboring coopera-degree, k, which is given by the average number oftors.If a defector is connected to k other indivi-links per individual (Ohtsuki et al., 2006). Thisdualsand jofthosearecooperators,then itspayoffrelationship can be shown with themethod ofis bj.Evolutionary dynamics are described by anpair-approximation for regular graphs, where allextremely simple stochastic process:at each timeindividuals have exactly the same number ofstep,a random individual adopts the strategy ofneighbors. Regular graphs include cycles, all kindsone of its neighbors proportional to their fitness.of spatial lattice, and random regular graphs.We note that stochastic evolutionary gameMoreover, computer simulations suggest that thedynamics in finite populations are sensitiveto therule b/c>k also holds for non-regular graphs suchintensity of selection.In general, thereproductiveas random graphs and scale-freenetworks.Thesuccess (fitness) of an individual is given by aruleholds inthelimitof weak selectionandk<N.constant, denoting the baseline fitness, plus theFor the completegraph,k=N, we always havepayoff that arises from the game under con-Pp>1/N>Pc-Preliminary studies suggest thatsideration.Strong selection meansthat thepayoffeqn 2.4 also tends to hold for strong selection.Theislarge compared withthebaselinefitness;weakbasic idea is that natural selection on graphs (inselectionmeansthepayoffissmall comparedwithstructured populations)can favor unconditionalthe baseline fitness.It turns out that many inter-cooperation without any need for strategic com-esting results can be proven for weak selection,plexity,reputation, orkin selection.Games on graphs grew out of the earlier tradi-which is an observation also well known inpopulation genetics.tion of spatial evolutionary game theory (NowakThe traditional, well-mixed population of evo-and May,1992;Herz,1994; Killingback andlutionary game theory is represented by the com-Doebeli,1996;MitteldorfandWilson,2000;Hauertplete graph, where all vertices are connected,etal.,2002;LeGalliard et al.,2003;Hauert andDoebeli, 2004; Szabo and Vukov, 2004) and inves-which means that all individuals interact equallyoften.In this special situation, cooperators aretigations of spatial models in ecology (Durrett andalways opposed by natural selection. This is theLevin, 1994a,1994b; Hassell et al.,1994; Tilman andfundamental intuition of classical evolutionaryKareiva,1997;Neuhauser,2001)andspatialmod-game theory.But what happens on other graphs?els in population genetics (Wright,1931;FisherWe need to calculate the probability, Pc, that aandFord,1950;Maruyama,1970;Slatkin,1981;single cooperator starting in a random positionBarton,1993;Pulliam,1988;Whitlock,2003)turnsthe wholepopulation fromdefectors intocooperators.If selection neither favors nor opposes2.5 Group selectioncooperation, then this probability is 1/N, which isthe fixation probability of a neutral mutant.If theThe enthusiastic approach of early group selec-fixation probability Pc is greater than 1/N, thentionists to explain all evolution of cooperationfrom this one perspective (Wynne-Edwards,1962)selection favors the emergence of cooperation.Similarly,we can calculate thefixation probabilityhas met with vigorous criticism (Williams, 1966)of defectors, Pp. A surprisingly simple rule deter-and even a denial of group selection for decades.mines whether selection on graphsfavors coopera-Onlyanembattled minorityof scientistscontinuedtion. Ifto studythe approach (Eshel,1972;Levin andKilmer,1974; Wilson,1975; Matessi and Jayakar,b/c>k(2.4)1976;Wade,1976;UyenoyamaandFeldman,1980;Slatkin,1981; Leigh,1983; Szathmary and Demeter,then cooperators have a fixation probability of1987).Nowadays it seems clear that group selectiongreater than 1/N and defectors have a fixationcan beapowerfulmechanismtopromote coopera-probability of less than 1/N.Thus, for graphtion (Sober and Wilson, 1998; Keller, 1999; Michod,selection to favor cooperation,the benefit/cost1999;Swensonetal.,2000;KerrandGodfrey-Smith,2002;ratio of the altruistic act must exceed the averagePaulsson, 2002; Boyd and Richerson, 2002; Bowles
can receive the benefit from neighboring cooperators. If a defector is connected to k other individuals and j of those are cooperators, then its payoff is bj. Evolutionary dynamics are described by an extremely simple stochastic process: at each time step, a random individual adopts the strategy of one of its neighbors proportional to their fitness. We note that stochastic evolutionary game dynamics in finite populations are sensitive to the intensity of selection. In general, the reproductive success (fitness) of an individual is given by a constant, denoting the baseline fitness, plus the payoff that arises from the game under consideration. Strong selection means that the payoff is large compared with the baseline fitness; weak selection means the payoff is small compared with the baseline fitness. It turns out that many interesting results can be proven for weak selection, which is an observation also well known in population genetics. The traditional, well-mixed population of evolutionary game theory is represented by the complete graph, where all vertices are connected, which means that all individuals interact equally often. In this special situation, cooperators are always opposed by natural selection. This is the fundamental intuition of classical evolutionary game theory. But what happens on other graphs? We need to calculate the probability, rC, that a single cooperator starting in a random position turns the whole population from defectors into cooperators. If selection neither favors nor opposes cooperation, then this probability is 1/N, which is the fixation probability of a neutral mutant. If the fixation probability rC is greater than 1/N, then selection favors the emergence of cooperation. Similarly, we can calculate the fixation probability of defectors, rD. A surprisingly simple rule determines whether selection on graphs favors cooperation. If b=c > k ð2:4Þ then cooperators have a fixation probability of greater than 1/N and defectors have a fixation probability of less than 1/N. Thus, for graph selection to favor cooperation, the benefit/cost ratio of the altruistic act must exceed the average degree, k, which is given by the average number of links per individual (Ohtsuki et al., 2006). This relationship can be shown with the method of pair-approximation for regular graphs, where all individuals have exactly the same number of neighbors. Regular graphs include cycles, all kinds of spatial lattice, and random regular graphs. Moreover, computer simulations suggest that the rule b/c > k also holds for non-regular graphs such as random graphs and scale-free networks. The rule holds in the limit of weak selection and k << N. For the complete graph, k ¼ N, we always have rD > 1/N > rC. Preliminary studies suggest that eqn 2.4 also tends to hold for strong selection. The basic idea is that natural selection on graphs (in structured populations) can favor unconditional cooperation without any need for strategic complexity, reputation, or kin selection. Games on graphs grew out of the earlier tradition of spatial evolutionary game theory (Nowak and May, 1992; Herz, 1994; Killingback and Doebeli, 1996; Mitteldorf and Wilson, 2000; Hauert et al., 2002; Le Galliard et al., 2003; Hauert and Doebeli, 2004; Szabo´ and Vukov, 2004) and investigations of spatial models in ecology (Durrett and Levin, 1994a, 1994b; Hassell et al., 1994; Tilman and Kareiva, 1997; Neuhauser, 2001) and spatial models in population genetics (Wright, 1931; Fisher and Ford, 1950; Maruyama, 1970; Slatkin, 1981; Barton, 1993; Pulliam, 1988; Whitlock, 2003). 2.5 Group selection The enthusiastic approach of early group selectionists to explain all evolution of cooperation from this one perspective (Wynne-Edwards, 1962) has met with vigorous criticism (Williams, 1966) and even a denial of group selection for decades. Only an embattled minority of scientists continued to study the approach (Eshel, 1972; Levin and Kilmer, 1974; Wilson, 1975; Matessi and Jayakar, 1976; Wade, 1976; Uyenoyama and Feldman, 1980; Slatkin, 1981; Leigh, 1983; Szathmary and Demeter, 1987). Nowadays it seems clear that group selection can be a powerful mechanism to promote cooperation (Sober and Wilson, 1998; Keller, 1999; Michod, 1999; Swensonet al., 2000; Kerr and Godfrey-Smith, 2002; Paulsson, 2002; Boyd and Richerson, 2002; Bowles 14 THEORETICAL ECOLOGY