E-CNN-LSTM Mind Wandering Detection Method on EEG and its Application
Li, Xiaolong (2023-04-19)
E-CNN-LSTM Mind Wandering Detection Method on EEG and its Application
(19.04.2023)
Julkaisu on tekijänoikeussäännösten alainen. Teosta voi lukea ja tulostaa henkilökohtaista käyttöä varten. Käyttö kaupallisiin tarkoituksiin on kielletty.
suljettu
Julkaisun pysyvä osoite on:
https://urn.fi/URN:NBN:fi-fe2023051945272
https://urn.fi/URN:NBN:fi-fe2023051945272
Tiivistelmä
In many classrooms of schools, intelligent teaching systems or products have already been used. Most of these systems are based on visual information technology or speech technology or bioelectric technology to collect the performance of students and teachers in the classroom. It is hoped that based on these performances, the system can judge whether students in the classroom are focused on listening. This article selects EEG signals as the data acquisition standard. This paper creatively applies the proposed EEG-CNN-LSTM deep neural network (abbreviated as E-CNN-LSTM) to wandering and focusing state detection system in class. "Mind wandering" is a concept in the field of psychology that reflects the shift in attention from task-related thoughts to task-unrelated thoughts. In the past decade, scientists have tried to find the changing rules between mind wandering and various biological signals. Among many biological signals, EEG signals are generally concealed, hard to imitate, and impossible to be generated by stress, compared with other biological characteristics such as human faces and speech signals.
On the topic of "exploring the relation between mind wandering state and EEG signals", researches based on deep learning methods at home and abroad are rare. In my opinion, a deep neural network is good at making feature extraction and solving classification problems. If a DNN is designed with a reasonable structure that corresponds to the biological characteristics of EEG signals, it can perform well. Based on this methodology, the main work done in this article can be listed as follows:
（1）Select an appropriate EEG data set of mind wandering experiment. Do reasonable data preprocessing according to the biological characteristics of the EEG signal.
（2）Propose a deep neural network structure based on convolutional neural network (CNN) and long short-term memory network (LSTM) at the aim of mind wandering detection. Different from other published experiments that artificially design features and use machine learning techniques to achieve classification, this paper completes the feature extraction and classification solution by deep neural networks. In other words, it achieves end-to-end training and detection. The first reason for its feasibility is that CNN is good at extracting spatial correlation information. The second is that LSTM
is good at extracting the characteristics of time series, and it is more sensitive to information a time step long before than a simple recurrent neural network. This advantage of LSTM is particularly critical for long sequences of EEG signals processed in this experiment. In addition, for the lack of training samples, some deep learning methods that reduce network overfitting are also used to improve the generalization ability of the network.
（3）Evaluate the performance of the E-CNN-LSTM model and make statistics of predictions. At the same time, the impact of the data preprocessing methods, the hyperparameter settings of the model, and the individual subjects are analyzed. Fine-tune based on the recognition results.
（4）Research the applications of E-CNN-LSTM. Propose a multi-dimensional database collection plan for smart classroom scenarios, and design a model output monitoring method for class focusing state recognition or identity recognition.
On the topic of "exploring the relation between mind wandering state and EEG signals", researches based on deep learning methods at home and abroad are rare. In my opinion, a deep neural network is good at making feature extraction and solving classification problems. If a DNN is designed with a reasonable structure that corresponds to the biological characteristics of EEG signals, it can perform well. Based on this methodology, the main work done in this article can be listed as follows:
（1）Select an appropriate EEG data set of mind wandering experiment. Do reasonable data preprocessing according to the biological characteristics of the EEG signal.
（2）Propose a deep neural network structure based on convolutional neural network (CNN) and long short-term memory network (LSTM) at the aim of mind wandering detection. Different from other published experiments that artificially design features and use machine learning techniques to achieve classification, this paper completes the feature extraction and classification solution by deep neural networks. In other words, it achieves end-to-end training and detection. The first reason for its feasibility is that CNN is good at extracting spatial correlation information. The second is that LSTM
is good at extracting the characteristics of time series, and it is more sensitive to information a time step long before than a simple recurrent neural network. This advantage of LSTM is particularly critical for long sequences of EEG signals processed in this experiment. In addition, for the lack of training samples, some deep learning methods that reduce network overfitting are also used to improve the generalization ability of the network.
（3）Evaluate the performance of the E-CNN-LSTM model and make statistics of predictions. At the same time, the impact of the data preprocessing methods, the hyperparameter settings of the model, and the individual subjects are analyzed. Fine-tune based on the recognition results.
（4）Research the applications of E-CNN-LSTM. Propose a multi-dimensional database collection plan for smart classroom scenarios, and design a model output monitoring method for class focusing state recognition or identity recognition.