09/06/2022
The Effects of Facial Attractiveness and Familiarity on Facial Expression Recognition
Jinhui Li1, Dexian He1, Lingdan Zhou1, Xueru Zhao2, Tingting Zhao3, Wei Zhang1,4,5 and Xianyou He1,4,5*
1School of Psychology, South China Normal University, Guangzhou, China
2Academy of Educational Science Talent Capital Base, Beijing Institute of Education, Beijing, China
3School of Health Management, Guangzhou Medical University, Guangzhou, China
4Center for Studies of Psychological Application, South China Normal University, Guangzhou, China
5Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, China
The classic theory of face perception holds that the invariant (e.g., identity and race) and variant (e.g., expression) dimensions of face information are independent of one another. Two separate neural systems are involved in face processing. However, the dynamic theory of face perception indicates that these two neural systems interact bidirectionally. Accordingly, by using the emotion categorization task and morph movies task, we investigated the influence of facial attractiveness on facial expression recognition and provided further evidence supporting the dynamic theory of face perception in both the static and dynamic contexts. In addition, this research used familiar celebrities (including actors, television personalities, politicians, and comedians) and explored the role of familiarity in face perception. In two experiments, the participants were asked to assess the expressions of faces with different levels of attractiveness and different levels of familiarity. We found that regardless of being in a static or dynamic face situation, happy expressions on attractive faces can be recognized more quickly, highlighting the advantage of happy expression recognition. Moreover, in static and dynamic familiar face situations, familiarity has a greater impact on expression recognition, and the influence of attraction on expression recognition may be weakened or even unaffected. Our results show that facial attractiveness influences the recognition of facial expressions in both static and dynamic contexts and highlight the importance of familiarity in face perception.
Introduction
Facial expressions can convey information regarding individuals’ emotions and social intentions, which is of great importance for social interaction. The rapid and correct identification of facial expressions is a necessity for successful social interaction. A classic cognitive model of face perception emphasizes the difference between the processes involved in the recognition of identity and those involved in the identification of expression (Bruce and Young, 1986). Based on this model, Haxby et al. (2000) proposed a model for the workings of this system that emphasized a difference between the indication of constant and variant sides of faces. The representation of the constant characteristics of faces (e.g., s*x, race, and identity) underlies the recognition of individuals, whereas the representation of the variant characteristics of faces (e.g., expression) underlies the perception of information that promotes social interaction.
In recent years, classic models of face perception have increasingly been challenged (Calder and Young, 2005; Hugenberg and Sczesny, 2006; Becker et al., 2007; Fisher et al., 2016). For example, one study found that the processing of facial identity and expression involves functional interactions and that their independence is not absolute (Calder and Young, 2005). This study proposed that the invariant and variable features of faces may be encoded by the same perceptual characterization system, followed by separation. Hugenberg and Sczesny (2006) found that participants could identify angry facial expressions faster in male faces than female faces. Becker et al. (2007) suggested that decisions regarding the gender of a face and facial expressions are not separate and found that subjects were faster and more accurate in discovering angry expressions on male faces and happy expressions on female faces. Fisher et al. (2016) identified the interaction between facial identity and expression. Similarly, other studies have concluded that there are different degrees of overlap between the brain regions that process face information (e.g., Ganel et al., 2005; Fox et al., 2009; Redfern and Benton, 2017). Specifically, Fox et al. (2009) found that the processes involved in facial identity and expression are not completely independent and that different degrees of overlap exist between the brain regions processing face information. Ganel et al. (2005) identified an interactive network responsible for the processing of expression and identity. Redfern and Benton (2017) used an identification task and concluded that expressions constitute a part of facial identity representation.
Given the debate regarding the classic theory of face perception, Quinn and Macrae (2011) proposed a dynamic theory of face perception. These authors proposed the existence of integrated processing pathways responsible for face processing. Facial characteristics (including invariant and variant characteristics) are processed in a multidimensional face coding system. The facial structure is coded in the primary stage; then, more sophisticated information is processed in the same dynamic system, and there is a general interaction. This view that facial characteristics are processed in a multidimensional face coding system has been confirmed by many studies (Ganel et al., 2005; Freeman et al., 2008; Fox et al., 2009). For example, Fox et al. (2009) found that facial identity and expression are not processed completely independently and that there are different degrees of overlap between the brain regions involved in face information processing. Other evidence suggesting that the invariant dimension of face information affects participants’ recognition of face’s variant dimensions has been reported (Craig et al., 2012; Fitousi and Wenger, 2013; Smith et al., 2017; Craig and Lipp, 2018).
Although previous research has typically focused on how invariant and variant face-related information is incorporated into judgments of facial expression, limited research has considered the relevance and importance of facial attractiveness. Face processing theories have paid minimal attention to the role of attractiveness and how attractiveness relates to other facial attributes. In the field of face perception, researchers have incorporated facial attractiveness into the invariant dimension of face information (Rhodes, 2006; Winston et al., 2007; Iaria et al., 2008). For example, Iaria et al. (2008) found that the fusiform gyrus (FFA) is activated when making facial attractiveness judgments and that the FFA mainly processes the invariant dimensions of faces. Additionally, noted attractiveness is based more on the temporally invariant aspects than the dynamic aspects of facial structure. Rhodes (2006) suggested that facial attractiveness may be more similar to the properties of identity and gender in terms of its processing demands. The attractiveness of a face is a salient social signal that reflects the overall effect of all physical attributes of a face.
Several studies have concluded that our perception of the attractiveness of a face is moderated by its facial expression (Magda and Goodwin, 2008; Tracy and Beall, 2011; Golle et al., 2014; Sutherland et al., 2017). In these studies, the participants perceived faces as more attractive when the facial expression was happy as opposed to other expressions. The apparent link between attractiveness and facial expression has been strengthened by recent neurological evidence emphasizing increased activity in the medial orbitofrontal cortex (OFC) during the presentation of stimuli that are attractive and positively valenced (O’Doherty et al., 2003). Sun et al. (2015) used the event-related potential (ERP) method to explore whether facial attractiveness and facial expression are processed similarly in the brain. They found that facial attractiveness and facial expression were separately embodied by two early components, i.e., N170 and P2, while their interaction effect was embodied by the late positive potential (LPP), which is a late component (Sun et al., 2015). Given that attractiveness is affected by facial expression recognition and that there is an overlapping brain region involved in facial attractiveness and facial expression recognition, we propose that attractiveness also affects expression recognition.
To the best of our knowledge, few studies have explored whether facial attractiveness contributes to facial expression, and the results of these studies are not consistent. Taylor and Bryant (2016) found that there was no interaction between facial attractiveness and expression. In their study, the authors asked the participants to categorize different facial expressions (happy, neutral, or angry) that varied with respect to facial attractiveness (attractive or unattractive). Their results suggested that facial attractiveness does not play a significant role in the judgment of happy or angry facial expressions. An earlier study also found no interaction between facial attractiveness and facial expression in the ratings of emotion valence (Jaensch et al., 2014). In contrast, Lindeberg et al. (2018) used an emotion category task and found that facial social classification cues influenced emotion perception. Thus, the authors found an interaction between facial attractiveness and expression. Specifically, they identified a greater happy face advantage resulting in more positively evaluated attractive faces than unattractive faces. Golle et al. (2014) indicated that the attractiveness of a face could affect the assessment of the happy expression. We suspect that these different experimental results may be caused by different experimental paradigms selected for different experiments. Taylor and Bryant (2016) and Lindeberg et al. (2018) used an emotional classification task in an experiment, but Lindeberg et al. (2018) used a larger sample size. Golle et al. (2014) utilized two alternative forced choice (2AFC) paradigms. It is also possible that different experiments use angry expressions as negative expressions and that angry expressions are often confused with other expressions (Taylor and Jose, 2014), leading to inconsistent conclusions in different studies. Although Lindeberg et al. (2018) verified that face attractiveness affects expression recognition, the findings of their study are inconsistent with the findings reported by Taylor and Bryant (2016). Therefore, more evidence concerning whether facial attractiveness affects facial expression identification should be collected. In addition, the facial expressions used in this research are happy and sad, which are not exactly the same as the happy and angry expressions used by Lindeberg et al. (2018). We used an experiment consistent with Lindeberg et al. (2018) in Experiment 1a. On the one hand, the paradigm investigates whether the recognition of facial expressions is affected by attractiveness. On the other hand, this study is an extension of existing research. The sad expression represents experimental material that expands the range of expressions affected by attractiveness and further verifies the relationship between facial attractiveness and expression recognition.
In addition, previous research illustrates that familiar stimuli prompt diverse positive reactions (Zajonc, 1968; Bornstein, 1989). Many studies have found that familiarity affects the processing of face perception (i.e., facial attractiveness and facial expressions) (Moreland and Beach, 1992; Dubois et al., 1999; Claypool et al., 2007; Carr et al., 2017; Yan et al., 2017). Moreover, studies have shown that there are strong interactions between familiarity and expression recognition (Claypool et al., 2007; Carr et al., 2017). For example, Carr et al. (2017) concluded that familiar faces appear happier and less angry than unfamiliar faces, indicating that familiarity affects facial expression recognition. Claypool et al. (2007) also found the same result. Furthermore, previous studies have examined how multiple social category cues, namely, s*x and race (Smith et al., 2017; Craig and Lipp, 2018) and s*x and age (Craig and Lipp, 2018), simultaneously moderate expression recognition and provided evidence of the combined influence of these social cues on expression recognition. However, no studies have investigated how facial attractiveness and familiarity simultaneously moderate expression recognition. Thus, in the present research, we manipulate facial familiarity.
More importantly, most existing research concerning facial expression recognition has used static face images (Claypool et al., 2007; Dobel et al., 2008; Carr et al., 2017), whereas in real life, faces are typically seen in motion. In addition, the dynamic context is more ecologically valid. That is, in interpersonal contexts, people’s facial expressions are usually in a dynamic situation (Niedenthal et al., 2000; Rubenstein, 2005; Ishii et al., 2011). Therefore, in this research, we presented both static and dynamic faces to subjects to judge facial expressions.
As mentioned above, the present research uses the emotion categorization task (see Bijlstra et al., 2010; Taylor and Bryant, 2016; Lindeberg et al., 2018) and morph movies task (see Niedenthal et al., 2000; Hugenberg and Bodenhausen, 2003; Bijlstra et al., 2014) in static and dynamic contexts. Accordingly, we investigate the extent to which attractiveness and familiarity influence facial expression processing. We conduct two experiments to explore this problem. According to the dynamic theory of face perception, if the attractiveness associated with face information can affect the processing of expression recognition, the processing of facial attractiveness and expression recognition are dependent on one another. However, according to the classic theory of face perception, if the facial attractiveness related to face information does not affect the processing of expression recognition, the processing of facial attractiveness and expression recognition are independent of one another. Based on behavioral evidence suggesting that attractive faces are often associated with positive personality characteristics (Dion et al., 1972; Golle et al., 2014; Wang et al., 2015; Lindeberg et al., 2018), we hypothesize that participants can recognize the happy expressions of attractive faces more quickly and that the advantages of happy expression recognition do not apply to unattractive faces in either a static context or a dynamic context (Experiments 1a and 1b). In addition, in accordance with previous studies (Carr et al., 2017; Smith et al., 2017; Craig and Lipp, 2018; Lindeberg et al., 2018), we anticipate that if familiarity has a greater impact on facial expression recognition, under the familiar face condition, the impact of attractiveness on facial expression recognition may be weakened or even unaffected. Similarly, compared to the other conditions, if familiarity and attractiveness together affect expression recognition, happy expressions on familiar attractive faces can be identified more quickly.
In general, the main aim of this research is to investigate whether facial attractiveness affects expression recognition in both static (Experiment 1a) and dynamic (Experiment 1b) contexts. This research also explores how familiarity and facial attractiveness can affect expression recognition in static (Experiment 2a) and dynamic (Experiment 2b) contexts.
