REVIEWS COGNITIVE NEUROSCIENCE OF HUMAN SOCIAL BEHAVIOUR Ralph Adolphs We are an intensely social species-it has been arqued that our social nature defines what makes us human,what makes us conscious or what gave us our large brains.As a new field. the social brain sciences are probing the neural underpinnings of social behaviour and have roduced a banquet of data that are both tantalizing and deeply puzzlng.We are finding new nmotion and reason,berveen actn and percepion andbets ner peo 0G NEUROSCIENCE But the包ls an ada gy and evol other:The fr ed by a neur app with studies of r s of hu an beha of ind dpy0 .th e secon that nate oa NATURE REVIEWS NEUROSCIENC VOLUME 4 MARCH 200315
REVIEWS A new field has emerged to investigate the cognitive neuroscience of social behaviour, the popularity of which is attested by recent conferences, special issues of journals1,2 and by books3,4. But the theoretical underpinnings of this new field derive from an uneasy marriage of two different approaches to social behaviour: sociobiology and evolutionary psychology on the one hand, and social psychology on the other. The first approach treats the study of social behaviour as a topic in ethology, continuous with studies of motivated behaviour in other animals. The second approach has often emphasized the uniqueness of human behaviour, and the uniqueness of the individual person, their environment and their social surroundings. These two different emphases do not need to conflict with one another. In fact, neuroscience might offer a reconciliation between biological and psychological approaches to social behaviour in the realization that its neural regulation reflects both innate, automatic and COGNITIVELY IMPENETRABLE mechanisms, as well as acquired, contextual and volitional aspects that include SELF-REGULATION. We share the first category of features with other species, and we might be distinguished from them partly by elaborations on the second category of features. In a way, an acknowledgement of such an architecture simply provides detail to the way in which social cognition is complex — it is complex because it is not monolithic, but rather it consists of several tracks of information processing that can be variously recruited depending on the circumstances. Specifying those tracks, the conditions under which they are engaged, how they interact, and how they must ultimately be coordinated to regulate social behaviour in an adaptive fashion, is the task faced by a neuroscientific approach to social cognition. Social cognition and emotion What is social cognition? If the social is ubiquitous, we face the problem of including all aspects of cognition as social. If it is special, we have to explain why and how (BOX 1). As a matter of practice, social brain science has indeed carved out a restricted domain of cognition. The bulk of studies emphasize motivational and emotional factors. Whereas other aspects of cognition — such as language, for example — contribute substantially to the regulation of social behaviour, the intuition has been that emotion stands in a privileged position. This intuition has its basis in our observations of other species and of human infants, whose social behaviour seems to be tightly coupled to emotion — a coupling that is heavily regulated in adults. But the intuition also has a functional explanation. Emotions can be thought of as states that coordinate homeostasis in a complex, dynamic environment; in so far as one aspect of the COGNITIVE NEUROSCIENCE OF HUMAN SOCIAL BEHAVIOUR Ralph Adolphs We are an intensely social species — it has been argued that our social nature defines what makes us human, what makes us conscious or what gave us our large brains. As a new field, the social brain sciences are probing the neural underpinnings of social behaviour and have produced a banquet of data that are both tantalizing and deeply puzzling. We are finding new links between emotion and reason, between action and perception, and between representations of other people and ourselves. No less important are the links that are also being established across disciplines to understand social behaviour, as neuroscientists, social psychologists, anthropologists, ethologists and philosophers forge new collaborations. COGNITIVELY IMPENETRABLE Processes that are not influenced strategically by cognition. They cannot be influenced at will, and their engagement is beyond our control. SELF-REGULATION The ability to control one’s behaviour effortfully and often in opposition to emotional drive (for example, controlling an anger outburst). Most prominent in adult humans, self-regulation depends on regions in the prefrontal cortex. NATURE REVIEWS | NEUROSCIENCE VOLUME 4 | MARCH 2003 | 165 Deparment of Neurology, University of Iowa, 200 Hawkins Drive, Iowa City, Iowa 52242, USA. e-mail: ralph-adolphs@uiowa.edu doi:10.1038/nrn1056 COGNITIVE NEUROSCIENCE
REVIEWS ox1 Are our brains specialized for social cognitior Hedgehog nd probably guide altruisti the left pre ntal nd a is to t diffe he r tation o timuli and the e th ral o the st ral motivation d in the c and the ohe and es nte are ngly adept ing reliab ures roughly in the same order as abov aint changes in facial expression',or a few seconds o 166 MARCH20O3 VOLUM正
MORAL EMOTIONS Guilt, shame, embarrassment, jealousy, pride and other states that depend on a social context. They arise later in development and evolution than the basic emotions (happiness, fear, anger, disgust, sadness) and require an extended representation of oneself as situated within a society. They function to regulate social behaviours, often in the long-term interests of a social group rather than the short-term interests of the individual person. MODULES Functional and/or anatomical components that are relatively specialized to process only certain kinds of information. Modules were originally thought of as cognitively impenetrable and informationally encapsulated (having restricted access to only certain information). Although most people do not view modules in such strict terms, there is evidence of domain-specific processing that is specialized for specific ecological categories (such as faces and social contract violations), although there is debate on this issue. EVENT-RELATED POTENTIALS (ERPs). Electrical potentials that are generated in the brain as a consequence of the synchronized activation of neuronal networks by external stimuli. These evoked potentials are recorded at the scalp and consist of precisely timed sequences of waves or ‘components’. MAGNETOENCEPHALOGRAPHY (MEG). A non-invasive technique that allows the detection of the changing magnetic fields that are associated with brain activity, similar to the detection of changing electric fields measured by ERPs. CATEGORIZATION Stimulus categories function to group together stimuli to which a similar behavioural response should be mounted. Coarse, generic categorization (for example, a dog as an animal) is superordinate; subordinate categorization includes basiclevel (a dog as a dog) and unique categories (a dog as your own pet). 166 | MARCH 2003 | VOLUME 4 www.nature.com/reviews/neuro REVIEWS Perception of social signals A large variety of stimuli are available for investigating social cognition (FIG. 2). Many recent studies on this topic have started, so to speak, at the input end — by showing pictures of social relevance to subjects (often under passive viewing conditions), and associating differences in the social content of stimuli with differences in the neural structures that are engaged in their processing. This work — primarily functional imaging studies — has found covariances between stimulus dimensions and brain structures. However, it is important to keep in mind that lesion studies are also needed to further elucidate a causal role for a given structure in a neural system (that is, to confirm that its role is essential). These lesion data are often lacking at this stage. It is also important to note that several of the structures that appear in this section will reappear later, reflecting their roles in implementing several social processes. Investigations have focused on the visual modality in primates, although a few studies have examined other sensory modalities as well (BOX 2). Social visual signals include information about the face (such as its expression and the direction of gaze), as well as about body posture and movement. Although prototypical facial expressions reliably signal the so-called basic emotions such as fear or happiness, human viewers are also surprisingly adept at making reliable judgements about social information from impoverished stimuli, such as faint changes in facial expression7 , or a few seconds of environment is social, emotions will participate in regulating social behaviour. In fact, one class of emotions — the so-called social or MORAL EMOTIONS — serve specifically in this capacity and probably guide altruistic helping5 and punishment6 . Most structures that have been shown to be important in processing emotions have therefore also turned out to be important for social behaviour. These include: first, specific regions in higher-order sensory cortices; second, the amygdala, the ventral striatum and orbitofrontal cortex; and third, additional cortical regions such as the left prefrontal, right parietal, and anterior and posterior cingulate cortices. It is possible to relate these three sets of regions to three different sets of processes. Higher-order sensory cortices are involved in the perceptual representation of stimuli and their constituent features. The amygdala, striatum and orbitofrontal cortex mediate an association of this perceptual representation with emotional response, cognitive processing and behavioural motivation. Higher cortical regions are then involved in the construction of an internal model of the social environment, involving representation of other people, their social relationships with oneself, and the value of one’s actions in the context of a social group. To some extent, these three sets of processes build on one another, although their interactions are complex (FIG. 1). For organizational purposes, the sections that follow consider these neural structures roughly in the same order as above. Box 1 | Are our brains specialized for social cognition? Brains and social behaviours vary across different mammalian species. Primitive insectivores (for example, hedgehogs) already show tightly regulated maternal behaviours that allow extended development of their offspring; non-human primates (for example, chimpanzees) live in extended societies of a few dozen subjects; and modern humans have created societies that encompass millions of interacting people. There is no question that humans are exceedingly skilled at large-scale social interaction, but it remains a puzzle how best to account for such abilities. Under one hypothesis149, the competition for social skills led to the evolution of cognitive mechanisms for outsmarting others150, and fuelled the expansion of the human brain and perhaps the elaboration of certain neural systems151. In support of this idea, there is a correlation across primate species between the size of their social group and the relative volume of neocortex149. Hedgehog Chimpanzee Human Courtesy of Laura Roberts
REVIEWS simulation of its.The fusifor us,the cted system of these The boeanatomicalinvestigatiosaecomplt sing EVENT-R (ERPs)an of face features.such a at is related to face-spec 6aepol ed s ategoriz ind d th of fa rith al id full-body interp process c study of f g order v s.as pes I 94 ing The sults point to a or the fus re rel tors of per al lob he s al gv an are such a nd motion,suc as gaze shifts' In line the role ofthe uperior tempor nd ventral vis mot more abstract n TURE REVIEWS
NATURE REVIEWS | NEUROSCIENCE VOLUME 4 | MARCH 2003 | 167 REVIEWS simulation of it15. The fusiform gyrus, the superior temporal gyrus and other less well specified regions of occipitotemporal cortex could therefore be thought of as an interconnected system of regions that construct a spatially distributed perceptual representation of different aspects of faces. There is good evidence that activation in all of these regions can be modulated by attention16 and by the context in which the visual social signal appears17,18. The above anatomical investigations are complemented by data on the timing of face processing. Studies using EVENT-RELATED POTENTIALS (ERPs) and MAGNETOENCEPHALOGRAPHY (MEG) show that some coarse CATEGORIZATION of face features, such as gender and emotion, can occur at latencies as short as 100 ms (REFS 19–22). Peak activity that is related to face-specific processing near the fusiform gyrus is seen around 170–200 ms (REF. 23). Whereas the construction of a detailed structural representation of the face therefore seems to require about 170 ms, some rapid, coarse categorization can occur with substantially shorter latencies, presumably indicating coarse perceptual routes that are parallel to a full structural encoding of the stimulus. A recent study24 investigated these different levels of categorization in detail and corroborated the idea of a fast, superordinate categorization of faces at a relatively short latency (around 100 ms). This categorization was followed by a subordinate categorization with a longer latency (around 170 ms), which was sufficient to discriminate individual identity. Similar evidence for the extraction of information at subordinate levels with increasing processing time has been provided by single-unit studies of face-selective cells in the monkey inferotemporal cortex25. At least three non-exclusive mechanisms could implement such computations: initial feed-forward processing followed by top–down modulation from higher regions, progressive processing within a region, or iterative cycles of processing between a region and others (either ‘lower’ or ‘higher’ in a processing hierarchy). A growing body of work has used visual stimuli that signal biological motion (FIG. 2) to study social cognition. Social psychologists first showed our propensity to make social inferences from visual motion of abstract shapes in the 1940s (REFS 26,27), and recent studies indicate that specific movement cues might generate attributions of ANIMACY, intentionality and AGENCY28,29. Visual motion stimuli elicit attributions of intentionality and animacy in infants, and robustly elicit intentional, emotional and personality attributions in adults, even when only static depictions of their trajectories are shown. More specific information about the movements of a human body are offered by POINT-LIGHT DISPLAYS30, which generate exceptionally robust shape-from-motion cues that allow the recognition of identity31, gender32, emotions33 and personality traits34. In line with the role of the superior temporal cortex in processing dynamic aspects of faces, this region is also activated by viewing biological motion in whole bodies35 or their point-light displays36,37, and by more abstract movements of geometric shapes38,39. full-body interpersonal interactions8 . Not only are we exceedingly sensitive to the social signals themselves, but we are also sensitive to the details of the context in which they occur. Regions of non-primary sensory cortices might be relatively specialized to process certain socially relevant stimulus attributes. The best evidence comes from the study of faces, for which higher-order visual cortices can be regarded as an assembly of MODULES that process distinct attributes, as borne out by various lesion studies, scalp and intracranial recordings, and functional imaging data. The results point to a role for the fusiform gyrus in processing the structural, static properties of faces, which are reliable indicators of personal identity. By contrast, regions more anterior and dorsal in the temporal lobe (such as the superior temporal gyrus and sulcus) are involved in processing information about the changeable configurations of faces, such as facial expressions, and eye and mouth movements9,10 (FIG. 3). Activation along the superior temporal sulcus and gyrus has been found when subjects view stimuli depicting biological motion, such as gaze shifts11,12 and mouth movements13. Processing in this region might draw on both dorsal and ventral visual streams in integrating shape and motion information14, and it might reflect a comparison of the observed action with the viewer’s ANIMACY The subjective impression that a stimulus is alive. AGENCY The subjective impression of a willful, goal-directed action. POINT-LIGHT DISPLAYS Visual motion stimuli created by attaching small lights to an actor’s body joints and filming the person moving in an otherwise dark room. Although they seem random when static, the biological motion of the lights immediately generates the compelling perception of a person moving about the room. Detailed perceptual processing • Fusiform gyrus • Superior temporal gyrus Emotional response in body • Visceral, autonomic, endocrine changes Social reasoning • Prefrontal cortex Coarse perceptual processing • Superior colliculus Selfregulation Reappraisal Modulation of cognition (memory, attention) • Cingulate cortex • Hippocampus • Basal forebrain Representation of emotional response • Somatosensory-related cortices Representation of perceived action • Left frontal operculum • Superior temporal gyrus Motivational evaluation • Amygdala • Orbitofrontal cortex • Ventral striatum Figure 1 | Processes and brain structures that are involved in social cognition. It is possible to assign sets of neural structures to various stages of information processing, as I argue in this review. However, the flow of social information defies any simple scheme for at least two reasons: it is multidirectional and it is recursive. A single process is implemented by a flexible set of structures, and a single structure participates in several processes, often during distinct windows of time. Processing routes differ in terms of their automaticity, cognitive penetrability, detail of the representations they involve and processing speed. The structures outlined in this figure share some core features of a social information processing system: selectivity (they make distinctions between different kinds of information), categorization and generalization, and the incorporation of past experience. Several of the components of social cognition (inside the grey shaded region) contribute to social knowledge. Reappraisal and self-regulation are particular modes of feedback modulation whereby evaluation and emotional response to social stimuli can be volitionally influenced
REVIEWS by:F. b).PEm d)and S Baron-Coben (e Box 2|Other sensory modalities used to study social cognition Most studies on nhave used that real-ife al pai of sl ot only as a function s that w s highly pro and the amygdal 168 MARCH 2003 VOLUME
168 | MARCH 2003 | VOLUME 4 www.nature.com/reviews/neuro REVIEWS These activations probably reflect the role of the superior temporal cortex in processing information about biological motion, on the basis of which we make social attributions. From perception to judgement Several brain regions are activated not only as a function of properties that are inherent to the stimuli, but also as a function of the psychological judgements that we make about them. In a sense, the influence of such judgements reflects a progressive decoupling from responses that are dictated by the stimulus itself to information that is generated by the brain through associations and inferences. The amygdala is one structure that is anatomically positioned to participate in such postperceptual processing, as it receives highly processed visual information from the anterior temporal cortices, and stores codes for subsequent processing of such perceptual information in other brain regions. In this way, it can influence memory, attention, decision making and other cognitive functions on the basis of the social significance of the stimuli that are being processed. a b d c e Figure 2 | Visual stimuli for investigating social cognition. These range from a | point-light walkers and b | dynamic geometric figures, to c | actual human social interactions. Facial expressions have been one of the most commonly used stimuli. d | They can be morphed from, say, fear to sadness and e | their eye region can signal specific social information, such as guilt, fear or flirtatiousness. Stimuli are from sets developed by: F. Heider (b), P. Ekman (d) and S. Baron-Cohen (e). Box 2 | Other sensory modalities used to study social cognition Most studies on social cognition have used visual stimuli, but it is clear that real-life social interactions draw on additional modalities. Whereas touch is an important social communication channel in other mammals, in modern humans it is relatively restricted to those with whom we have the most intimate relationships. A recently described distinct neural pathway of slow-conducting, C-afferent fibres that convey information about pleasant, light touch to the insula could underlie processing of social somatosensory signals, such as a caress152. The sense of smell provides powerful social signals in other mammals but, again, it seems to be less important in humans. Laboratory studies have found influences of odorants on human physiology, but the effects of odours on social behaviour are less clear. Whereas the orbitofrontal cortex and the amygdala are activated by the emotional quality of odours in humans153,154, and pheromones differentially activate the human hypothalamus155, the links of these findings to actual social behaviour remain unclear. Audition provides important social signals in addition to language. The intonation of speech — prosody — can signal various emotions, and is recognized using some of the same structures that we use for recognizing facial expressions156. Music is an especially intriguing stimulus, as it might serve a social function that is not found in other animals, and it has been shown to elicit intense emotional responses that activate the orbitofrontal cortex, the insula and the amygdala157
REVIEWS and attracti hthe ch on the n the bly ion nal pr that a don and v onal gu ts per An impo tion iss en early s gardles ith rave r pro the ext On th her ha rather than the phy of th to pe dtanandco on in the ri cture parti is attract whi can be ma mpled tails of th ask and attractive the more average o OLUME 4 MARCH 2003160
NATURE REVIEWS | NEUROSCIENCE VOLUME 4 | MARCH 2003 | 169 REVIEWS Judging race, trustworthiness and attractiveness Beyond its role in recognition of basic emotions, the amygdala is involved in more complex social judgements. For example, it shows differential habituation of activation to faces of people of another race54, and its activation has been found to correlate with race stereotypes of which the viewer might be unaware55. However, the role of the amygdala in processing information about race is still unclear. Other brain regions in the extrastriate visual cortex are also differentially activated as a function of race56, and lesions of the amygdala do not seem to impair race judgements57. Other kinds of social judgement also seem to involve the amygdala. In one study, patients with bilateral amygdala damage were found to be impaired in judging how much to trust another person from viewing their face. They all judged other people to look more trustworthy and more approachable than did normal viewers58, a pattern of impairment that is also consistent with the often indiscriminately friendly behaviour of such patients in real life (FIG. 4a). The role of the amygdala in processing stimuli related to potential threat or danger therefore extends to the complex judgements on the basis of which we approach or trust other people. These lesion studies have been complemented by functional imaging studies on the role of the amygdala in judging trustworthiness (FIG. 4b). When normal subjects view faces of people that look untrustworthy, activation is found in the superior temporal sulcus, the amygdala, the orbitofrontal cortex and the insular cortex59, perhaps outlining a sequence of processes that encompass perception, judgement and aspects of emotional response. Interestingly, some activation of the amygdala by untrustworthy-looking faces is independent of factors such as gender, gaze, race or emotional expression of the face59. Given that much of the variance in the physical dimensions of different faces can be eliminated yet still produce amygdala activation, it is possible to assume that this activation reflects the judgements and inferences that subjects make about the face, rather than its perceptual properties. An important future direction will be to examine the variance in viewers’ personality traits in these social judgements, as has been done in two recent studies correlating amygdala activation to emotional expressions with viewers’ extraversion60 or anxious temperament due to a POLYMORPHISM in the serotonin transporter promoter61. To the extent that the amygdala activation covaries with differences in the personality of the viewer, rather than the physical composition of the stimulus, we can conclude that we are tapping processes more distal to perception and closer to judgement, decision making, and the interpersonal behaviours that are based on them. Another class of social judgement that we make from faces is attractiveness, which can be manipulated by specific properties of faces. For instance, faces are perceived to look more attractive the more average or symmetrical they are, or with greater exaggeration of robusticity and NEOTENY features, all of which have been proposed to signal differential fitness. Moreover, such preferences by women can vary across different phases The bulk of research on the human amygdala has used emotional facial expressions as stimuli and has pointed most consistently to this region being involved in the processing of fear and related emotions40–42, although recent evidence indicates that its role is probably much broader43,44. Functional imaging studies show processing at several stages: a rapid, automatic evaluation and tagging of stimuli for further processing16, feedback modulation of attentional processing in visual cortices45, and modes of processing that are subject to self-regulation and volitional guidance46,47. The first and last of these stages show complementary roles for the amygdala, probably operating at complementary timescales. On the one hand, some amygdala activation is seen early48, regardless of the conscious perception of the stimulus (for example, in response to subliminal stimuli49,50 or in patients with BLINDSIGHT51 or hemispatial NEGLECT52), and regardless of attention allocation in some tasks16. On the other hand, effortful self-regulation of the emotions induced by stimuli47, REAPPRAISAL of their emotional importance46 and difficult attentional tasks53, all modulate amygdala activation. These findings urge caution in the rigid assignment of cognitive processes to neural structures, because it is probable that a given structure participates in several processes, depending on the time at which its activity is sampled and on the details of the task and context. It is conceivable that the amygdala participates both in the initial, rapid evaluation of the emotional significance of stimuli, and in a later assessment within a given context and goal. BLINDSIGHT The ability of a person with a lesion in the primary visual cortex to reach towards or guess at the orientation of objects projected on the part of the visual field that corresponds to this lesion, even though they report that they can see nothing in that part of their visual field. NEGLECT A neurological syndrome (often involving damage to the right parietal cortex) in which patients show a marked difficulty in detecting or responding to information in the contralesional field. REAPPRAISAL Reinterpretation of a situation to assign it a different value. Whereas reappraisal changes emotional response by changing one’s perception of the stimulus, other strategies of self-regulation directly modulate emotional response despite one’s original perception. POLYMORPHISM The simultaneous existence in the same population of two or more genotypes in frequencies that cannot be explained by recurrent mutations. Lateral fusiform gyrus Inferior occipital gyrus Superior temporal sulcus Figure 3 | Activation in visual cortices to viewing faces. Changeable, dynamic aspects of faces, such as expression and gaze, activate the superior temporal sulcus, whereas static aspects activate the fusiform gyrus. The top panel shows these activations on a human brain smoothed to reveal both sulci (darker grey) and gyri. The bottom panel shows a flattened representation of the same data. Data were generated by contrasting the activations to viewing faces with those to viewing houses (orange, greater activation to faces; blue, greater activation to houses). Modified, with permission, from REF. 10 © (2000) Elsevier Science