August 29, 2024
Editors' notes
This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:
fact-checked
trusted source
proofread
Evolutionary mental state transition model helps machine learning algorithms track emotions
Seeking to improve automatic emotion tracking, which detects and monitors emotions over time, a group of researchers in the field of human-computer interaction decided to approach the task by modeling shifts in internal emotions rather than only interpreting external emotional signals.
Using insights from psychology, they developed the evolutionary mental state transition model, a model that incorporates a mental state transition network. They tested its effectiveness on two multimodal emotion datasets, producing noticeably more accurate results than existing alternatives.
Their research was published on April 8, 2024 in Intelligent Computing.
In addition to accuracy, another advantage of the evolutionary mental state transition model for emotion tracking is its reduced computational time and smaller footprint. The model has fewer parameters than other published models, which makes it "suitable for deployment on mobile devices and robots," according to the authors.
Daily life applications of emotion tracking include public opinion monitoring, marketing communications, mental health monitoring, and online education. Extensions of the authors' model could be developed to personalize emotion tracking to take into account individual variations in emotional fluctuation. Work in this direction would build on the psychologically realistic nature of the model, which attempts to capture the "natural dynamics of emotions and their impact on mental states."
The authors' system for emotion tracking consists of several steps:
- Multi-modal pattern recognition based on language, vision, and acoustic inputs
- Feature fusion in a transformer
- Pooling to calculate "external emotional energy" (apparent emotion)
- Determination of actual emotion using a unique mental state transition network
In the evolutionary mental state transition model, language, vision, and acoustic features are first extracted from the data and encoded, retaining their chronological order. Next, multihead cross-attention blocks are used to fuse the features at each time step; this is the most computationally intensive step. Third, maximum pooling and average pooling, two varieties of a common deep learning technique, are used for dimensionality reduction, and the features are transformed into external emotional energy at each time step.
Finally, the mental state transition network is used to take into account patterns in changes in the subject's emotions over time, as well as external emotional energy, to determine the actual emotional state at a particular time step.
The network was built on a set of probabilities resulting from data previously collected from 200 participants about the associations between different emotion pairs. It predicts emotional state in part by weighing the contributions of multiple simultaneous emotions rather than assuming a subject is experiencing only one.
The performance of the evolutionary mental state transition model was compared with that of a number of baseline methods using classification tasks based on two large datasets, the CMU Multimodal Opinion Sentiment and Emotion Intensity dataset and the Ren Chinese Emotion Corpus. The CMU dataset, consisting of recorded monologues in English, identifies happiness, sadness, anger, disgust, surprise, and fear. The Chinese corpus consists of blog texts, and was used to test the mental state transition network component.
More information: Fu-Ji Ren et al, Tracking Emotions Using an Evolutionary Model of Mental State Transitions: Introducing a New Paradigm, Intelligent Computing (2024). DOI: 10.34133/icomputing.0075
Provided by Intelligent Computing Citation: Evolutionary mental state transition model helps machine learning algorithms track emotions (2024, August 29) retrieved 29 August 2024 from https://techxplore.com/news/2024-08-evolutionary-mental-state-transition-machine.html This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
Explore further
Generating empathetic machine responses through emotion tracking and constraint guidance shares
Feedback to editors
