In recent decades, human emotions related research studies are given more attention in neuroscience and affective Computing [1]. In our daily life emotions plays a crucial role, we as human can comprehend the emotions of different people yet it is truly inconceivable for computers to do likewise. In earlier decades researchers have investigated emotion detection through text [2], speech [3], facial expressions and gestures [4–7]. However, the disadvantages of using these techniques are that if someone conceals their feelings [8], the results are vague and this turns out to be a question mark on their reliability. Currently, electroencephalography (EEG) signals are the promising way to implement communication between a person and machine. Different studies have been conducted in the literature, so far, on emotion classification based on EEG signals, which yields significant growth in terms of accuracy [9,10]. The main difference lies in terms of signal acquisition, noise reduction, features extraction and classification techniques that are used. In [11,12], the authors investigate processing algorithms and pattern classification of EEG signals respectively which produce better results. In [13] the authors have used different physiological signals along with mobile data to detect human emotion. Results show better detection however, the labeling of the emotion is not done using audio or visual feedback. In [14], the authors describe importance of human emotions recognition. For this purpose, the authors consider pictures to predict human emotions recognition using traditional EEG (13 electrodes). Moreover, six human emotions are classified using neural network classifier. In [15], the authors introduce recognition of eight human emotions, whether these emotion(s) get matched of the required specifications using EEG (3 electrodes). The international affective picture system (IAPS) stimuli is employed in their proposed methodology, and it is observed that K-nearest neighbors (KNN) classifier gives best accuracy rate. In [16], the authors discuss the human emotions recognition by considering three emotions. Music soundtrack is assumed in their study for emotions elicitation and for classification of emotions support vector machine (SVM) is considered. The authors in [17] use multiclass SVM, entropy and higher order autoregressive model for features extraction. In [18], EEG data is recorded through EEG neuroscan with a sampling frequency of 1000 Hz, power spectral density (PSD) technique is employed for features extraction and SVM with a Gaussian kernel for classification. In [19], the authors investigate offline analysis of EEG through SVM with a poly kernel and high order crossings (HOC) for features extraction. The Authors In [20], proposed an emotion recognition system, in which they employed discrete Fourier transform (DFT), wavelet transform (WT) and gabor techniques for dimensionality reduction and probabilistic neural network (PNN) were employed as a classifier. The authors in [21] record signals through Emotiv (2048 Hz) and KNN (k = 4) is used as a classifier. Differential entropy and rational asymmetry are calculated as features and videos is used as a stimuli [22]. In [23], six emotions are classified using music videos (1 min); IAPS (40 s) as stimuli, signals are acquired from five subjects using nervus EEG (256 Hz), band pass frequency filter of 0.2–45 Hz; common average referencing (CAR) is applied to EEG data followed by Daubechies wavelet transform (Db4) and multilayered perceptron neural networks with backpropagation algorithm (MLP-BP) as a classifier. In [24], music clips (30 s each) are used as stimuli, data is acquired from 26 subjects using EEG neuroscan (500 Hz), band pass filter of 1–100 Hz followed by fast fourier transform (FFT) for features extraction on all considered bands i.e., delta δ, theta θ, alpha, α, beta, β, and gamma, γ, of EEG signals and then SVM with radial basis function (RBF) kernel is employed as a classifier. In a recent work Fang et al. in [25] proposed a multi featured emotion classification method based on EEG however the accuracy is very low and also the method does not recognize cross-subject emotions. In [19], the acquisition of the signals is carried out using g.MOBIlab (256 Hz) from 16 subjects, the Ekman’s picture set (5 s) as a stimuli, band pass filter of 8–30 Hz; CAR applied to EEG data followed by higher order crossings (HOC) as a feature is used and SVM with polynomial kernal is considered as a classifier. In [26], a user-independent study is described where signals are acquired using neurosky mindwave headset (512 Hz) from thirty subjects, the authors use music clips as stimuli, band pass filter of 1–50 Hz; down sampling to 300 Hz followed by statistical features, power spectral density (PSD), FFT, and WT as features extraction methods and MLPBP is used as a classifier. Due to high computational cost and lack of accuracy when applied in signals corrupted by colored Gaussian noise, most of the above methods are not robust. Some of the above methods focus on detection of a specific emotion while others are application dependent. Most of these methods are also based on features extracted in one specific domain which are not robust. This research proposes a novel method of human emotions classification based on EEG signals acquired during audiovisual stimuli. EEG is useful tool for emotions classification as it depicts emotions in real-time which can’t be changed deliberately. Furthermore, we used a novel strategy of recording EEG signals and human emotions using audiovisual stimuli. After recording the EEG signals and constructing a new dataset, the EEG signal is then processed and twelve different features are extracted from it in three domains namely time, frequency and wavelet. RF and MLP are then used as classifiers for classifying human emotions. Our contributions of this works are summarized as follows: 1)

A new dataset of EEG signal and the labeled human emotion is developed using a novel approach. The EEG signals and human emotions are recorded in response to audiovisual stimuli using neurosky mindwave headset (NMH) device. In other studies the researcher have either used audio or visual stimuli but both stimuli are not used simultaneously while recording the EEG and human emotion.

The remainder of this paper is organized as follows: In Section 2, Materials and methods is briefly presented. In Section 3, we present the results and discussion. Finally, the concluding remarks with futuristic directions are provided in Section 4.

The EEG based emotions classification approach is shown in Fig. 1, the steps are described as; 1)

The subject (participant) is exposed to the stimulus and voltage differences of the subject mind are recorded,

All these steps are briefly discussed in the following subsections.

An offline analysis of EEG based emotions classification has been explored in this paper. EEG signals are captured for 40 subjects. All signals are recorded with the help of simple and low cost single channel NMH with a sampling frequency of 512 Hz. Zero DC-offset signals are attained by mean subtraction method. Twelve features of pre-processed signals are assumed. The features are further divided into four groups; group first consists of hybrid features, group second contains time domain features, group third consists of frequency domain features and group four consists of wavelet domain features as shown in Tab. 1.

The MLP and RF classifiers are then trained using 10-fold cross validation on each group. the classification accuracy on each group of the features set is computed. as shown in Tab. 2. After the comparison of classifiers output for all four feature sets, it is concluded that features in Set 1 are the best features for the proposed work. RF is contributing best results for Set 1. The performance factors in our work are avg. accuracy rate, F-measure, Mean absolute error and kappa statistic are shown in Tab. 3. Accuracy means ratio of truly classified to the total samples. The F-measure or F − 1 is harmonic mean of the precision and recall which is a measure of a test’s accuracy, F1 score for each emotion is shown in Tab. 4. Mean absolute error is the average of absolute errors, can better reflect the actual situation of the prediction error.

Kappa coefficient measure classification accuracy, the calculation of the kappa coefficient is based on the confusion matrix. The kappa calculation result is [−1, 1], but usually kappa falls between [0, 1], which can be divided into five groups to represent different levels of consistency: [0.0, 0.20] very low consistency (slight), [0.21, 0.40] Fair, [0.41, 0.60] moderate, [0.61, 0.80], and [0.81, 1] are almost perfect.

The confusion matrices of MLP and RF are shown in Tabs. 5 and 6 respectively. From the results of the proposed method it is evident that the sad and happy emotions are successfully classified with high accuracy. However, the accuracy of anger, fear and surprise emotions are not much as. To sum up, as compared to other emotions happy and sad are easy to recognize while watching video clips.

Tab. 7 provides the summary of the EEG signals used in this study for human emotions classification. The signals acquired from different subjects are used to evaluate the performance of the classification system, which is based on stimuli used, emotions, equipment (frequency) type, method for feature extractions, and type of classifier. Comparative analysis of MLP and RF classifiers using time, frequency, wavelet and hybrid features are illustrated in Fig. 6.

In this paper, we analyzed recognition of human’s emotions based on EEG signals using visual stimuli. Twelve selected features in time, frequency and wavelet domains are extracted and then used as input to the classifier. Five emotions namely anger, fear, happy, sad and surprise are classified using RF and MLP classifiers. From the results obtained for several subjects data, it is evident that the RF classifies emotions with highest accuracy.

Although the proposed model produced better results but improvement can still be made. One of the strands relates directly to the hardware such as multi-channel EEG device can be used for obtaining more accurate results. However, the acquisition of new equipment entails an increase in cost. The emotional induction is one of the most influential criteria in the emotional behavior of the subject and in the respective responses provided at the biological level, so that the improvement of multimedia content is one of the considerations to be taken into account for possible future work.