The Truth About Facial Coding: What You Need to Know for Marketing Research

Introduction

‍

Facial coding, which involves the often AI-based analysis of facial expressions to infer emotional states, has received significant attention in the field of marketing research. However, as with any methodology, facial coding is not without its limitations and controversies. In particular, the theoretical underpinnings of the technique have recently come under critical scrutiny. 

‍

In this blog post, we aim to provide a comprehensive overview of the current debate surrounding facial coding in both academic and marketing research. We will also highlight some of the potential pitfalls and limitations that practitioners should be mindful of when using this methodology. 

‍

Our goal is to provide you with a nuanced understanding of the pros and cons of facial coding, as well as guidance on how to use it responsibly to gain deeper insights into consumer attitudes and behaviors.

‍

The Pros of Facial Coding in Marketing Research

‍

Facial coding has been praised by both agencies offering products with this method and clients using it for its positive aspects in detecting emotions. Here are some of the key benefits of facial coding: 

‍

1. Non-invasive: Facial coding does not require any physical contact or the use of special equipment, which makes it a convenient and user-friendly method for measuring potential emotional responses.

‍

2. Real-time: Its ability of real-time measurements can be useful for studying reactions to various stimuli, such as ads or packaging. This is believed to be able to provide valuable insights into how consumers respond to different marketing messages and can help in the development of more effective marketing campaigns.

‍

3. Quantitative: Facial coding provides a quantitative measure of emotional responses, which can help identify key emotional triggers that drive consumer behavior, leading to develop more effective marketing strategies.

‍

4. Objective: Facial coding is designed to provide an objective measure of emotional responses. This can be useful in reducing bias and increasing data reliability.

‍

5. Cost-effective: It does not require expensive equipment or specialized training. This makes it a more accessible research method for smaller brands or organizations with limited resources.

‍

6. Remote data collection: The data for facial coding can be collected remotely, which is particularly useful in today’s age of social distance and online shopping. This can also provide a more naturalistic environment for studying emotional responses to marketing stimuli. 

‍

The Controversy Surrounding Facial Coding in the Academic Literature

‍

Despite the touted benefits described above, facial coding has also been the subject of criticism from both academics and practitioners. In order to use the method responsibly, it is important to be aware of these criticisms.  

‍

Historically, the idea of facial coding originated with the observations of American psychologist Paul Ekman in the 1960s and 1970s, who suggested that facial expressions were obligatory movements inherent to primate evolution, implying that emotional expressions are universal¹. These assumptions went largely unchallenged for several decades, until a new cohort of psychologists and cognitive scientists began to revisit the topic and question the conclusions.

‍

Lisa Feldman Barrett has been one of the most vocal critics of facial coding. She argues that the method fails to recognize the role of context and individual differences in emotion perception. According to Barrett, emotions are not fixed and universally recognized categories. Rather, they are socially constructed and can vary across cultures and individuals. Therefore, facial expressions may not always accurately reflect an individual's emotional state or intent.

‍

Barrett also suggested that facial expressions may not always be the best indicator of emotional experience, as individuals may be able to control their facial expressions to present a certain image or hide their true emotions. In addition, she has argued that emotional experience involves more than just facial expressions and may also include physiological changes, thoughts, and other cognitive processes that are not directly observable from the outside.

‍

The criticism of the universality of facial expressions was supported by a comprehensive review of over 1,000 scientific papers². The review showed that facial expressions vary substantially across cultures, situations, and even between people in the same situation. The authors concluded that there is little to no evidence that people can reliably infer another person’s emotional state from a set of facial movements. This review has shaken the very foundations of the concept of facial coding.

Practical Limitations and Criticisms of Facial Coding

‍

Consistent with the skepticism from new generation scientists, some organizations have begun to question the usefulness of facial coding for detecting emotions and have even withdrawn from developing or offering emotion recognition technology. 

‍

For example, the AI Now Institute, a research center at New York University, has called for a ban on the use of emotion recognition technology in sensitive situations such as hiring or law enforcement³. Some tech giants, including Microsoft and Google, have also faced controversy over their facial recognition software, including its ability to identify emotions⁴. 

‍

In the marketing research industry, NielsenIQ, one of the leading agencies offering consumer neuroscience research, recently announced that they no longer use facial coding technology in any of their solutions, including their renowned ad testing toolkit. This decision was based on their extensive analysis of facial expression data from over 2,000 video ad tests across 15 countries⁵. 

‍

There are several practical limitations and concerns behind this withdrawal trend. Some of these are as follows:

‍

1. Lack of standardization: One of the main criticisms of facial coding is that there is a lack of standardization in the measurement and interpretation of facial expressions. Different researchers and companies may use different coding systems or interpret facial expressions differently, which can make it difficult to compare results across studies.

‍

2. Limited predictive power: While facial coding may provide insights into consumers' emotional responses to stimuli, it may not necessarily predict actual behavior. Emotions that can be inferred from facial coding are only a minor factor influencing consumer decisions, and they may not always accurately reflect consumers' true emotions or intentions.

‍

3. Cultural differences: Facial expressions can vary across different cultures, making it difficult to apply facial coding universally. What may be interpreted as a "happy" expression in one culture may not be interpreted in the same way in another culture.

‍

4. Ethical concerns: Some critics of facial coding have raised ethical concerns about the use of facial recognition technology in marketing research. These concerns include issues around privacy, data security, and the potential for misuse or abuse of the technology.

‍

5. Overreliance on neuroscience: Finally, some critics argue that facial coding and other neuroscience-based research methods may place too much emphasis on individual-brain-based explanations of behavior, while neglecting the role of other factors such as social and environmental influences.

‍

Caveats and Responsible Use of Facial Coding in Marketing Research

‍

Despite the criticisms and concerns surrounding facial coding in both academia and industry, many tech companies and agencies continue to offer software to identify emotions. However, academic researchers on both sides of the debate remain skeptical of this type of software due to concerns about the data used to train algorithms and the ongoing debates around the science⁶.

As a practitioner, it's critical for you to keep all of these limitations and criticisms in mind when using this technology with a critical eye. Failure to do so could lead to inaccurate or misleading results, as well as ethical concerns. Some of the key points for using this method are as follows:

‍

1. Recognize the limitations of facial coding: You should be aware of the limitations of facial coding and its potential for producing false positives or negatives, especially in complex emotional contexts. Reconsider its necessity, notwithstanding the inherent risks discussed above.

‍

2. Use facial coding in conjunction with other methods: Facial coding should not be used as the sole method for capturing emotional responses, but rather as one tool in a larger research toolkit. Using multiple methods can help to validate and triangulate findings.

‍

3. Be aware of cross-cultural differences: Practitioners should be aware of cross-cultural differences in facial expressions and be cautious when generalizing findings across different populations.

‍

4. Address ethical concerns: Ethical concerns regarding privacy and the use of personal information should be addressed. Best practices for data protection should also be followed.

‍

5. Seek expert advice: Practitioners should seek the critical advice from experts, ideally a neutral third party, in facial coding and/or neuroscience to ensure that they are using the method correctly and interpreting the results accurately. Don’t blindly trust what agencies and software vendors claim. 

‍

In conclusion, while facial coding can be a valuable tool for understanding consumer responses, it is essential to approach its use in marketing research with a critical and responsible eye. If not acknowledged and accounted for, the risks of unreliable data and flawed conclusions can arise, leading to ineffective marketing strategies.

‍

References

1. Ekman P, Sorenson ER, Friesen WV (1969) Pan-cultural elements in facial displays of emotion. Science, 164: 86 – 88.
2. Barrett LF et al. (2019) Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20: 1 – 68.
3. Crawford K et al. (2019) AI now 2019 report. New York, NY: AI Now Institute
4. https://www.analyticsinsight.net/google-and-microsoft-call-emotion-ai-risky-but-only-limits-usage/
5. https://nielseniq.com/global/en/insights/commentary/2022/about-face-the-shift-away-from-facial-coding-technology/
6. Heaven D (2020) Why faces don’t always tell the truth about feelings. Nature, 578: 502 – 504.