Home Datasets Products Publications & Code Contact

MPHA Dataset


Multimodal Psychological Health Analysis Dataset

  Overview of the Dataset

    Psychological disorders have become a global health concern, affecting the quality of life and social adaptability of millions of people. However, current diagnostic methods primarily rely on subjective judgment by clinicians, lacking objective and reliable biomarkers. This results in inconsistencies in diagnostic standards, inefficiency, and insufficient accuracy. With the rapid development of artificial intelligence technologies, using multimodal data for intelligent diagnosis and assessment of psychological disorders shows significant innovative potential and application value.
    This dataset was constructed through systematically designed emotion stimulation experiments, collecting four modalities of data: facial video, audio, electromyography (EMG), and electroencephalography (EEG). It encompasses a wide range of emotional and physiological responses. The dataset employs diverse emotional stimuli (e.g., watching videos, describing images, reading words, and listening to audio) for experiments with different groups, aiming to identify the most effective stimuli for eliciting emotional responses. It not only lays the foundation for uncovering the complex mechanisms of psychological disorders but also provides critical data support for the early detection and precise diagnosis of psychological disorders across different populations.



Figure 1. Data Overview

  Main components of Datasete


The dataset currently covers four types of multimodal information. As shown in Figure 2, each sample consists of the following three main components:


Figure 2. Data Structure Diagram

  Basic Attributes

        This section contains the fundamental identifying information for each sample, consisting of a unique Index to distinguish different samples and a basic Label for identification

  Data content

This is the core of the dataset, gathering multiple modalities of experimental data. It specifically includes Video (visual information), Audio (sound information), EMG (Electromyography signals reflecting muscle activity), and EEG (Electroencephalogram signals recording brain activity).


Table 1. Attributes of Multimodal Data

Figure 3. Data Visualization

  Data Labels

This section provides the definitive classification for each data sample. Labels are categorized as either Normal or Depression, determined by the scores obtained from the Beck Depression Inventory-II (BDI-II) self-assessment scale.

Copyright © 2024 All rights reserved.
Yunnan Key Laboratory of Software Engineering
Yunnan University, China.