Various-information coupling emotion recognition method for human-computer interaction

一种面向人机交互的多类信息耦合的情感识别方法

Abstract

本发明公开了一种基于深度学习的多类信息耦合的情感识别方法,其特征是按如下步骤进行:1获取人脸表情的视频数据以及语音数据;2对文本内容进行文本特征提取,获得文本信息特征;3提取语音数据的韵律学特征和整体语音特征并进行耦合,获得语音信息特征;3对视频数据进行进行图像特征提取,获得表情信息特征;4对文本信息特征、语音信息特征和表情信息特征进行耦合,获得综合信息特征;5利用深度学习方法对综合信息特征进行数据优化,并利用分类器对优化的综合信息特征进行训练,获得情感识别模型,以情感识别模型实现对多类信息耦合的情感识别。本发明能全面结合文本、语音和视频三个方面的数据信息,从而提高人机交互中的情感状态判断的准确度。
The invention discloses a various-information coupling emotion recognition method for the human-computer interaction. The method is characterized by including the steps of 1, acquiring the video and audio data of facial expression; 2, extracting features of text content, and acquiring the text information features; 3, extracting and coupling the prosodic features and overall audio features of the audio data; 4, coupling the text information features, audio information features and expression information features, and acquiring the comprehensive information features; 5, performing data optimization on the comprehensive information features by the deep learning method, utilizing a classifier to train the optimized comprehensive information features, and acquiring an emotion recognition model for various information coupling emotion recognition. According to the method, data information of text, audio and video can be combined completely, and the accuracy of emotion state judgment in human-computer interaction can be improved accordingly.

Claims

Description

Topics

Download Full PDF Version (Non-Commercial Use)

Patent Citations (5)

    Publication numberPublication dateAssigneeTitle
    CN-101261832-ASeptember 10, 2008北京航空航天大学汉语语音情感信息的提取及建模方法
    CN-101685634-AMarch 31, 2010上海盛淘智能科技有限公司Children speech emotion recognition method
    CN-103164691-AJune 19, 2013深圳市金立通信设备有限公司System and method for recognition of emotion based on mobile phone user
    CN-103198827-AJuly 10, 2013合肥工业大学Voice emotion correction method based on relevance of prosodic feature parameter and emotion parameter
    US-2011310237-A1December 22, 2011Institute For Information IndustryFacial Expression Recognition Systems and Methods and Computer Program Products Thereof

NO-Patent Citations (4)

    Title
    LOUIS-PHILIPPE MORENCY等: "“towards multimodal sentiment analysis:Harvesting opinions from the web”", 《PROCEEDINGS OF THE 13TH INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES.ACM》
    LOUIS-PHILIPPE MORENCY等: "“utterance-level multimodal sentiment analysis”", 《PROCEEDINGS OF THE 51ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS》
    徐永华等: "“语音识别系统中多种特征参数组合的抗噪性”", 《金陵科技学院学报》
    曲利新: "“基于深度信念网络的语音情感识别策略”", 《中国优秀硕士学位论文全文数据库 信息科技辑》

Cited By (1)

    Publication numberPublication dateAssigneeTitle
    CN-104881685-ASeptember 02, 2015清华大学基于捷径深度神经网络的视频分类方法