Deteksi Indikasi Kelelahan Menggunakan Deep Learning

Dhomas Hatta Fudholi(1*), Royan Abida N Nayoan(2), Maghfirah Suyuti(3), Ridho Rahmadi(4),

(1) Jurusan Informatika, Universitas Islam Indonesia
(2) Jurusan Informatika, Universitas Islam Indonesia
(3) Jurusan Informatika, Universitas Islam Indonesia
(4) Jurusan Informatika, Universitas Islam Indonesia
(*) Corresponding Author

Abstract


Many students experience fatigue due to lack of sleep which can be caused by a psychological conditions or bad habits. Lack of sleep can affect student’s performance academically and causes many illnesses, stress and depression. Students with fatigue causes students to not study well, increasing risk of academic failure and will lead to having low GPA. In this research, fatigue detection is carried out to find out which students are experiencing fatigue. In this study, an annotated video dataset was used with a total of 18 subjects acted drowsy and alert. Fatigue detection is based on mouth movements, therefore mouth annotation is used. Mouth annotation has 2 categories, namely annotation 0 which indicates a closed mouth and annotation 1 which indicates the mouth is yawning. Previous study proves ResNet50 has better performance than other pre-trained models such as AlexNet, Clarifia, VGG-16, and GoogLeNet-19. We also applied image augmentation which is useful for providing new image variations to the model in each epoch by changing the rotation, random shift, and random zoom. ResNet50 model is used to perform binary classification which has two outputs, namely mouth stillness and yawning. The results of the frame classification are evaluated using precision, recall and f1-score. By using ResNet model, the results of the classification of frames labeled 0 or mouth stillness, obtained a precision of 0.72, a recall of 0.88, and an f1-score of 0.79. Meanwhile, the frame classification labeled 1 or yawning has a precision value of 0.85, a recall of 0.65, and an f1-score of 0.74.

Full Text:

PDF

References


M. A. Alsaggaf, S. O. Wali, R. A. Merdad, and L. A. Merdad, “Sleep quantity, quality, and insomnia symptoms of medical students during clinical years: Relationship with stress and academic performance,” Saudi Med. J., vol. 37, no. 2, pp. 173–182, 2016, doi: 10.15537/smj.2016.2.14288.

G. Maheshwari and F. Shaukat, “Impact of Poor Sleep Quality on the Academic Performance of Medical Students,” Cureus, vol. 11, no. 4, pp. 3–8, 2019, doi: 10.7759/cureus.4357.

M. Z. Alom et al., “A state-of-the-art survey on deep learning theory and architectures,” Electron., vol. 8, no. 3, pp. 1–67, 2019, doi: 10.3390/electronics8030292.

J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2009, vol. 20, no. 11, pp. 248–255, doi: 10.1109/CVPR.2009.5206848.

K. W. Kim, H. G. Hong, G. P. Nam, and K. R. Park, “A study of deep CNN-based classification of open and closed eyes using a visible light camera sensor,” Sensors (Switzerland), vol. 17, no. 7, 2017, doi: 10.3390/s17071534.

M. Ngxande, J. R. Tapamo, and M. Burke, “Detecting inter-sectional accuracy differences in driver drowsiness detection algorithms,” arXiv, 2019.

U. Lahoti, R. Joshi, N. Vyas, K. Deshpande, and S. Jain, “Drowsiness detection system for online courses,” Int. J. Adv. Trends Comput. Sci. Eng., vol. 9, no. 2, pp. 1930–1934, 2020, doi: 10.30534/ijatcse/2020/158922020.

T. P. Nguyen, “Eye Tracking System to Detect Driver Drowsiness,” pp. 472–477, 2015.

R. Ahmad and P. J. N. Borole, “Drowsy Driver Identification Using Eye Blink detection,” vol. 6, no. 1, pp. 270–274, 2015.

M. S. Devi and P. Bajaj, “Driver Fatigue Detection Using Mouth and Yawning Analysis,” no. January 2008, 2015.

C. Yan, F. Coenen, Y. Yue, X. Yang, and B. Zhang, “Video-Based Classi ¯ cation of Driving Behavior Using a Hierarchical Classi ¯ cation System with Multiple Features,” vol. 30, no. 5, pp. 1–33, 2016, doi: 10.1142/S0218001416500105.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-December, pp. 770–778, 2016, doi: 10.1109/CVPR.2016.90.

C. Goutte and E. Gaussier, “A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation,” in Lecture Notes in Computer Science, vol. 3408, no. April, 2005, pp. 345–359.

K. Tjong, E. F. Sang, and F. De Meulder, “Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition,” 2003, doi: 10.3115/1118853.1118877.

N. Srivastava, G. Hinton, A. Krizhevsky, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” 2014.

C. Weng, Y. Lai, S. Lai, “Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network”, In Asian Conference on Computer Vision Workshop on Driver Drowsiness Detection from Video, 2016.




DOI: http://dx.doi.org/10.30645/j-sakti.v5i1.292

Refbacks

  • There are currently no refbacks.


J-SAKTI (Jurnal Sains Komputer & Informatika)
Published Papers Indexed/Abstracted By:

Jumlah Kunjungan :

View My Stats