  chernyatiev-al-lebedev-ap-vestnik-pip-2021-4.pdf |
- 27
- Digital emotion analytics: a pilot study of human emotion recognition using mobile device sensors
- Chernyatiev A.L., Lebedev A.P. Digital emotion analytics: a pilot study of human emotion recognition using mobile device sensors. Vestnik of Kostroma State University. Series: Pedagogy. Psychology. Sociokinetics, 2021, vol. 27, № 4, pp. 200-207. (In Russ.) https://doi.org/10.34216/2073-1426-2021-27-4-200-207
- DOI: https://doi.org/10.34216/2073-1426-2021-27-4-200-207
- УДК: 159.93
- Publish date: 2021-11-28
- Annotation: Affective computing is a rapidly developing area at the intersection of psychology and the development of artificial intelligence systems. At the moment, systems for recognising human emotions from photos and videos (facial expressions), voice recordings (intonation), gestures, posture, gait and other data using various machine learning algorithms are actively developed. This research project is devoted to the recognition of emotions and mental states of a person through the sensors of mobile devices (accelerometer, gyroscope, etc.), which record the features of micro- and macro-hand motor skills characteristic of the states under study. The pilot study considered the possibility of fixing and differentiating psychoemotional states according to the readings of sensors of mobile devices (tablet, smartphone), using machine learning models. As a result, it was possible to obtain models that determine, according to the readings of the sensors of a mobile device in the hands of a person, whether it is in a neutral emotional state or under stress. Moreover, it was possible to differentiate the state of stress according to two modalities – stress caused by psychological reasons ("fulfillment of obligations") and psychophysiological reasons (unpleasant noise in the headphones). The statistically significant differences, as well as the relatively high accuracy of the constructed machine learning model, allow us to speak about the reliability of the results obtained, and they confirm the hypothesis about the possibility of identifying and classifying emotional states using the sensors of mobile devices.
- Keywords: affective computing, emotion recognition, stress, diagnostics, mobile device sensors, machine learning
- Funding and acknowledgments: The work was carried out with the support of the Fund for Assistance to Innovation, project GRNTIS5/63391
- Literature list: Krjukova T.L. Psihologija sovladajushhego povedenija: sovremennoe sostojanie i psihologicheskie, sociokul'turnye perspektivy [Psychology of coping behavior: current state and psychological, socio-cultural perspectives]. Vestnik Kostromskogo gosudarstvennogo universiteta [Bulletin of the Kostroma State University], 2013, vol. 19/5, pp. 184–488. (In Russ.) Kurganskij N.A., Nemchin T.A. Ocenka psihicheskoj aktivacii, interesa, jemocional'nogo tonusa, naprjazhenija i komfortnosti [Assessment of mental activation, interest, emotional tone, tension and comfort]. Praktikum po jeksperimental'noj i prikladnoj psihologii: ucheb. posobie [Workshop on experimental and applied psychology]. L., 1990, pp. 44–50. (In Russ.) Rasskazova E.I., Gordeeva T.O. Koping-strategii v psihologii stressa: podhody, metody i perspektivy issledovanij [Coping strategies in the psychology of stress: approaches, methods and prospects for research]. Psihologicheskie issledovanija: jelektronnyj nauchnyj zhurnal [Psychological research: electronic scientific journal], 2011, vol. 3, pp. 1–4. (In Russ.) Hazova S.A. Koping-resursy subekta: osnovnye napravlenija i perspektivy issledovanija [Coping resources of the subject: main directions and prospects of research]. Vestnik Kostromskogo gosudarstvennogo universiteta [Bulletin of the Kostroma State University], 2013, vol. 19/5, pp. 188–191. (In Russ.)