Mental Health Diagnosis Using CNNs and Vision Transformers
CNN–Vision Transformer research for binary classification of Bengali voice samples, published in Journal of Voice.
Journal article · 2024
Workflow at a glance
Prepare voice data
Represent Bengali recordings as spectrograms.
Learn CNN + ViT features
Study local and broader acoustic patterns.
Classify study labels
Predict the binary stable/unstable labels.
Evaluate research results
Assess study metrics and interpretation.
Project facts & reported results
Reported result
Reported study result; see linked paper for evaluation protocol.
Reported result
Reported study result; see linked paper for evaluation protocol.
Implementation fact
Project Overview
Problem Statement
Small voice datasets and acoustic variation make mental-health-related classification difficult to evaluate reliably. This study explores spectrogram learning in the Bengali voice domain.
Approach & Methodology
Converted audio to log-mel spectrograms, treating voice analysis as a vision-driven learning problem. Proposed hybrid CNN-ViT architecture: CNN layers extract local acoustic patterns, ViT captures long-range dependencies in temporal sequences. Applied class imbalance handling (SMOTE), data augmentation, and robust evaluation methodology. Used ethically collected Bengali voice data from mental health institutions in Bangladesh.
Outcome & Scope
Published a CNN–Vision Transformer study in Journal of Voice, reporting approximately 91% accuracy and 0.97 ROC-AUC. Contributed a Bengali voice dataset for research use.
System Architecture
Hybrid CNN-ViT pipeline for voice-based mental health classification
Key Components:
Noise reduction, normalization, and feature extraction (mel-spectrograms, MFCCs)
Convolutional layers capturing local acoustic patterns and spectral features
Self-attention mechanism for long-range temporal dependencies in voice sequences
Binary classification of the study labels: stable and unstable.
Interpretability module showing which audio segments contribute to predictions
Key Features & Capabilities
Audio preprocessing pipeline with noise reduction and consistent 48 kHz sampling
Log-mel spectrogram conversion (spectrogram-as-image paradigm)
Hybrid CNN-ViT architecture for feature learning
Class imbalance handling with SMOTE
Attention visualization for interpretability
Cross-validation with robust evaluation methodology
Current Scope & Limitations
Research use only. These experiments do not establish a clinically validated diagnostic or screening tool.
Reported scores apply to the study protocol. Split definitions and evaluation details should be checked in the linked paper before comparing results across studies.