Home / Insights / Is facial coding accurate?

Is facial coding accurate? What the research says (2026)

Facial coding is accurate at one job and less certain at another. Automated systems are good at detecting facial movements, the Action Units of the Facial Action Coding System, such as the lip-corner pull of a smile. They are less certain at saying what a person feels, because the same movement can accompany different emotions in different people and situations. For ad testing, the evidence is encouraging when results are averaged across many viewers. A 2023 study of 219 people watching video commercials found that automatic facial coding predicted their self-reported emotion and their ratings of the ads and brands, and added information beyond what they reported. A large webcam study of 170 ads linked facial responses to ad liking and, less strongly, to purchase intent. The limits are real: people watching alone show little expression, webcams and lighting vary, and a face is not a direct readout of feeling. Use it across a sample, alongside a survey.

Updated 17 October 2026 by Jonathan Prescott, Founder, Cavefish.

What “accurate” means here

“Is facial coding accurate?” is really two questions, and they have different answers.

Can software detect facial movements? This is a measurement question. The Facial Action Coding System (FACS) describes every visible facial movement as a numbered Action Unit, such as AU12, the lip corner puller. Automated coders are generally reliable on strong, common movements like smiles, and less reliable on subtle ones or in poor conditions.

Can those movements tell you what someone feels? This is an inference question, and it is where the scientific debate sits. A smile usually goes with enjoyment, but people also smile when they are embarrassed, polite or uncomfortable. The same face can mean different things in different contexts.

Ad testing needs both, but the second matters most. The useful question is not whether one face is read perfectly but whether facial response, averaged across a sample, tells you something true about how an ad lands.

The evidence for facial coding in advertising

Three peer-reviewed studies are most often cited, and they point the same way.

Höfling and Alpers (2023)

Researchers at the University of Mannheim showed video commercials to participants and coded their faces automatically. In their words, facial expressions “significantly predicted self-report of emotion as well as advertisement and brand effects”, and had “incremental value beyond self-report of emotion”. In other words, the face told the researchers something the questionnaire did not. They analysed 219 people; 38 of the 257 who took part were excluded because their faces could not be detected well enough. Read the paper.

McDuff and colleagues (2015)

A team from MIT and Affectiva collected more than 12,000 webcam facial responses from 1,223 people to 170 ads, coded frame by frame. They report that ad liking could be predicted from facial responses with good accuracy, and that changes in purchase intent were predicted less accurately. They also found that strong positive responses that came straight after the brand appeared were more likely to be effective. Read the paper.

Teixeira, Wedel and Pieters (2012)

This Journal of Marketing Research study combined automated facial expression detection with eye tracking and recorded when viewers zapped away from ads. Joy and surprise both helped hold attention and keep people watching, in different ways: the level of surprise mattered most for attention, and how quickly joy rose mattered most for keeping viewers. Read the paper.

The limits you should know

  • Faces are not a direct readout of feelings. A major 2019 review led by Lisa Feldman Barrett tested the common belief that facial movements reliably reveal emotions against four criteria: reliability, specificity, generalisability and validity. It concluded that inferring a specific emotion from a facial movement is far less straightforward than is often assumed. Read the review.
  • Regulators are cautious. In October 2022 the UK Information Commissioner's Office warned organisations to assess the risks before using emotion analysis technologies, and described the market as immature. ICO biometrics reports; coverage.
  • People watching alone are not very expressive. Most viewers show little visible expression for most of an ad. Results depend on aggregating the moments that do move faces.
  • Conditions vary. Lighting, glasses, beards, head angle and webcam quality all affect detection. In the Höfling and Alpers study, about one in seven participants had to be excluded for face-detection problems.
  • Culture shapes display. Norms for showing emotion differ between cultures, so compare results within a market.

How to use facial coding well

  • Aggregate. Read the response of a sample, not individual faces. EchoDepth recommends 15 or more viewers per audience segment.
  • Pair it with what people say. A short survey after viewing shows where stated and felt responses agree and where they split.
  • Insist on transparency. Ask any provider which Action Units drive a score and how the score is calculated. A number you can't trace back is hard to trust.
  • Check the exclusions. A good report says how many viewers started, how many were excluded and why.
  • Treat it as a diagnostic. Facial coding is strongest at showing where in an ad the response rises or falls. Treat any claim that it forecasts sales with care unless the provider shows the evidence.

EchoDepth publishes its pipeline, the Emotional Response Curve calculation and its limits in the methodology. For what a single smile does and doesn't tell you, see AU12 and the Duchenne smile.

Questions

Not with certainty for one person at one moment. Software can reliably detect many facial movements, but the same movement can go with different feelings. Facial coding is most useful when responses are averaged across many viewers and paired with what they say.

Peer-reviewed studies have used webcam and automated facial coding to predict how much people like ads and brands. Webcam quality and lighting vary, so a good study checks each viewer's camera before the ad plays and reports how many viewers were excluded.

Research links facial responses to ad liking and, less strongly, to purchase intent. Direct links to sales are harder to show. EchoDepth has not yet published validation of its Emotional Response Curve against sales, so it presents the curve as a diagnostic of response.

EchoDepth recommends 15 or more viewers per audience segment for each ad or edit, because individual faces are noisy and the signal is in the average. Every group you want to report on separately is its own segment and needs its own 15.

In the UK and EU it can be used with informed consent from research participants. The EU AI Act bans emotion recognition in workplaces and education. EchoDepth only codes the faces of consenting research viewers and does not store video.

References

  1. Höfling, T. T. A., & Alpers, G. W. (2023). Automatic facial coding predicts self-report of emotion, advertisement and brand effects elicited by video commercials. Frontiers in Neuroscience, 17. doi:10.3389/fnins.2023.1125983
  2. McDuff, D., el Kaliouby, R., Cohn, J. F., & Picard, R. W. (2015). Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads. IEEE Transactions on Affective Computing, 6, 223–235. doi:10.1109/TAFFC.2014.2384198
  3. Teixeira, T., Wedel, M., & Pieters, R. (2012). Emotion-induced engagement in internet video advertisements. Journal of Marketing Research, 49(2), 144–159. doi:10.1509/jmr.10.0207
  4. Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. doi:10.1177/1529100619832930
  5. Information Commissioner's Office (2022, October). Biometrics: insight and foresight reports. ico.org.uk

See it on your own ad

EchoDepth pre-tests video ads on real viewers and reports the response second by second, with every score traced back to the Action Units behind it.

Test an adHow ad testing worksRead the methodology