Physiology-based emotion recognition while using an everyday electronic device

Capturing positive and negative emotions while applying technology is a complex, but profitable subject. Positive emotions improve the desirability of a product and lead to an enhanced probability of purchase and recommendation (positive UX). Today, we rely predominantly on data obtained from subjective questionnaires (e. g. How did you feel about this product? How are you feeling right now?) - even though much of what we do is processed unconsciously and our responses to such questions can be biased due to social desirability. The consideration of physiological information of users is a promising field to orient future product developments to their end users and thus to be able to design them according to their needs.

As important and profitable as the study of emotions during technology use is, the more complex and difficult questions arise: How does one recognize emotions on a physiological level? How does one capture the physiological data in real-world usage scenarios? How does one evaluate the data in a realistic way and derive understandable and tangible interpretations from it? We have addressed these questions in a project in collaboration with a global technology company. 

Scientifically grounded and state-of-the-art recording methods were selected to record emotions during technology use: Electroencephalography, electrocardiography and measurement of sweat formation on the skin (skin conductance) were collected as physiological parameters. Facial processing software, which uses machine learning methods, was used to automatically decode emotions based on facial movements (so-called facial decoding). In addition, subjective data were collected by questionnaires to compare the experimental set-up with the participants' responses.

Analyses of the multiple physiological and subjective data collected were performed using machine learning and higher statistics, and interpretations were derived and presented in an understandable way for the general public. 

This project reveals that positive and negative emotions during technology use can be captured and distinguished from each other - both in physiology and facial movement analysis. These real-life measurements could be obtained using machine learning, and thus providing new insights for our clients. The results are valid, reliable and scientifically at the highest level. 

Through insights from physiology-based emotion recognition studies, customers can gain important insights into the attractiveness, satisfaction and acceptance of their product without relying solely on the subjective statements of the testers. Thus, our multi-method approach provides a more comprehensive insight and understanding into the user's experience of technical products.