Micro-Expression Recognition Using the Spatiotemporal Feature extraction and Deep-Learning Methods

[1] V. Esmaeili, M. Mohassel Feghhi, and S. O. Shahdi, "A comprehensive survey on facial micro-expression: approaches and databases," Multimedia Tools and Applications, Vol. 81, No. 28, pp. 40089-40134, 2022.

[2] X. Li et al., "Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods," IEEE transactions on affective computing, Vol. 9, No. 4, pp. 563-577, 2017.

[3] G. Zhou et al., "Micro-expression action unit recognition based on dynamic image and spatial pyramid," The Journal of Supercomputing, pp. 1-24, 2023.

[4] Y. Li, J. Wei, Y. Liu, J. Kauttonen, and G. Zhao, "Deep learning for micro-expression recognition: A survey," IEEE Transactions on Affective Computing, 2022.

[5] G. Zhao, X. Li, Y. Li, and M. Pietikäinen, "Facial Micro-Expressions: An Overview," Proceedings of the IEEE, 2023.

[6] P. Ekman and W. V. Friesen, "Nonverbal leakage and clues to deception," Psychiatry, Vol. 32, No. 1, pp. 88-106, 1969.

[7] P. Ekman, Micro expressions training tool Emotionsrevealed. com, 2003.

[8] V. Esmaeili, M. Mohassel Feghhi, and S. O. Shahdi, "Automatic Micro-Expression Recognition Using LBP-SIPl and FR-CNN," AUT Journal of Modeling and Simulation, Vol. 54, No. 1, pp. 59-72, 2022.

[9] V. Esmaeili, M. Mohassel Feghhi, and S. O. Shahdi, "Autonomous apex detection and Micro-expression recognition using proposed diagonal Planes," International Journal of Nonlinear Analysis and Applications, Vol. 11, pp. 483-497, 2020.

[10] V. Esmaeili, M. M. Feghhi, and S. O. Shahdi, "Micro-expression recognition using histogram of image gradient orientation on diagonal planes," in 2021 5th International Conference on Pattern Recognition and Image Analysis (IPRIA), 2021: IEEE, pp. 1-5.

[11] V. Esmaeili and S. O. Shahdi, "Automatic micro-expression apex spotting using Cubic-LBP," Multimedia Tools and Applications, Vol. 79, pp. 20221-20239, 2020.

[12] V. Esmaeili, M. M. Feghhi, and S. O. Shahdi, "Automatic micro-expression apex frame spotting using local binary pattern from six intersection planes," arXiv preprint arXiv:2104.02149, 2021.

[13] V. Esmaeili, M. Mohassel Feghhi, and S. O. Shahdi, "Micro-Expression Recognition based on the Multi-Color ULBP and Histogram of Gradient Direction from Six Intersection Planes," Journal of Iranian Association of Electrical and Electronics Engineers, Vol. 19, No. 3, pp. 123-130, 2022.

[14] V. Esmaeili, M. M. Feghhi, and S. O. Shahdi, "Early COVID-19 Diagnosis from Lung Ultrasound Images Combining RIULBP-TP and 3D-DenseNet," in 2022 9th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS), 2022: IEEE, pp. 1-5.

[15] V. Esmaeili and M. Mohassel Feghhi, "Real-time Authentication for Electronic Service Applicants using a Method Based on Two-Stream 3D Deep Learning," Soft Computing Journal, 2023.

[16] V. Esmaeili and M. Mohassel Feghhi, "Diagnosis of Covid-19 Disease by Combining Hand-crafted and Deep-learning Methods on Ultrasound Data," Journal of Machine Vision and Image Processing, Vol. 9, No. 4, pp. 31-41, 2022.

[17] V. Esmaeili, M. Mohassel Feghhi, and S. O. Shahdi, "Spotting micro‐movements in image sequence by introducing intelligent cubic‐LBP," IET Image Processing, Vol. 16, No. 14, pp. 3814-3830, 2022.

[18] V. Esmaeili and M. M. Feghhi, "COVID-19 Diagnosis:ULBPFP-Net Approach on Lung Ultrasound Data," Iranian Journal of Electrical & Electronic Engineering, Vol. 19, No. 3, 2023.

[19] V. Esmaeili, M. Mohassel Feghhi, and S. Shahdi, "Applying Partial Differential Equations on Cubic Uniform Local Binary Pattern to Reveal Micro-Changes," Journal of Electrical and Computer Engineering Innovations (JECEI), pp. 259-270, 2023.

[20] A. Ebrahimi, S. Luo, and R. Chiong, "Introducing transfer learning to 3D ResNet-18 for Alzheimer’s disease detection on MRI images," in 2020 35th international conference on image and vision computing New Zealand (IVCNZ), 2020: IEEE, pp. 1-6.

[21] Z. Raisi and J. Zelek, "Investigation of Deep Learning Optimization Algorithms in Scene Text Detection," International Journal of Industrial Electronics Control and Optimization, Vol. 6, No. 3, pp. 171-182, 2023. and Optimization, Vol. 6, No. 3, pp. 171-182, 2023.

[22] R. Mehta and K. Egiazarian, "Rotated local binary pattern (RLBP): rotation invariant texture descriptor," in 2nd International Conference on Pattern Recognition Applications and Methods, ICPRAM 2013, Barcelona, Spain, 15.-18.2. 2013, 2013, pp. 497-502.

[23] T. Ojala, M. Pietikäinen, and T. Mäenpää, "A generalized local binary pattern operator for multiresolution gray scale and rotation invariant texture classification," in Advances in Pattern Recognition—ICAPR 2001: Second International Conference Rio de Janeiro, Brazil, March 11–14, 2001 Proceedings 2, 2001: Springer, pp. 399-408.

[24] T. Pfister, X. Li, G. Zhao, and M. Pietikäinen, "Recognising spontaneous facial micro-expressions," in 2011 international conference on computer vision, 2011: IEEE, pp. 1449-1456.

[25] H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand,and W. Freeman, "Eulerian video magnification for revealing subtle changes in the world," ACM transactions on graphics (TOG), Vol. 31, No. 4, pp. 1-8, 2012.

[26] A. Asthana, S. Zafeiriou, S. Cheng, and M. Pantic, "Robust discriminative response map fitting with constrained local models," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 3444-3451.

[27] W.-J. Yan, Q. Wu, Y.-J. Liu, S.-J. Wang, and X. Fu, "CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces," in 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG), 2013: IEEE, pp. 1-7.

[28] W.-J. Yan et al., "CASME II: An improved spontaneous micro-expression database and the baseline evaluation," PloS one, Vol. 9, No. 1, p. e86041, 2014.

[29] J. Li et al., "CAS (ME) 3: A third generation facial spontaneous micro-expression database with depth information and high ecological validity," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, No. 3, pp. 2782-2800, 2022.

[30] J. Tang, L. Li, M. Tang, and J. Xie, "A novel micro-expression recognition algorithm using dual-stream combining optical flow and dynamic image convolutional neural networks," Signal, Image and Video Processing Vol. 17, No. 3, pp. 769-776, 2023.

[31] X.-B. Nguyen, C. N. Duong, X. Li, S. Gauch, H.-S. Seo, and K. Luu, "Micron-bert: Bert-based facial micro-expression recognition," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1482-1492.

[32] H. Yang, S. Sun, and J. Chen, "Deep Learning-Based Micro-Expression Recognition Algorithm Research," International Journal of Computer Science and Information Technology, Vol. 2, No. 1, pp. 59-70, 2024.

[33] C. Nicholson, "Evaluation metrics for machine learning accuracy, precision, recall, and F1 defined,"Pathmind.http://pathmind.com/wiki/accuracy-precision-recall-f1, 2019.

[34] S.-T. Liong, J. See, K. Wong, and R. C.-W. Phan, "Less is more: Micro-expression recognition from video using apex frame," Signal Processing: Image Communication, Vol. 62, pp. 82-92, 2018.

[35] Y. S. Gan, S.-T. Liong, W.-C. Yau, Y.-C. Huang, and L.-K. Tan, "OFF-ApexNet on micro-expression recognition system," Signal Processing: I mage Communication, Vol. 74, pp. 129-139, 2019.

[36] S.-T. Liong, Y. S. Gan, J. See, H.-Q. Khor, and Y.-C. Huang, "Shallow triple stream three-dimensional cnn (ststnet) for micro-expression recognition," in 2019 14th IEEE international conference on automatic face & gesture recognition (FG 2019), 2019: IEEE, pp. 1-5.

[37] Y. Gan, S.-E. Lien, Y.-C. Chiang, and S.-T. Liong, "LAENet for micro-expression recognition," The Visual Computer, Vol. 40, No. 2, pp. 585-599, 2024.

[38] Z. Wang, K. Zhang, W. Luo, and R. Sankaranarayana, "HTNet for micro-expression recognition," arXiv preprint arXiv:2307.14637, 2023.

[39] Y. Su, J. Zhang, J. Liu, and G. Zhai, "Key facial components guided micro-expression recognition based on first & second-order motion," in 2021 IEEE International Conference on Multimedia and Expo(ICME), 2021: IEEE, pp. 1-6.

[40] M. Wei, W. Zheng, Y. Zong, X. Jiang, C. Lu, and J. Liu, "A novel micro-expression recognition approach using attention-based magnification-adaptive networks," in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022: IEEE, pp. 2420-2424.

[41] H. Jin, N. He, Z. Li, and P. Yang, "Micro-expression recognition based on multi-scale 3D residual convolutional neural network," Mathematical Biosciences and Engineering, Vol. 21, No. 4, pp. 5007-5031, 2024.

[42] B. Song et al., "Recognizing spontaneous micro-expression using a three-stream convolutional neural network," Ieee Access, vol. 7, pp. 184537-184551, 2019.

[43] B. Sun, S. Cao, D. Li, J. He, and L. Yu, "Dynamic micro-expression recognition using knowledge distillation," IEEE Transactions on Affective Computing, Vol. 13, No. 2, pp. 1037-1043, 2020.

[44] Y. Wang et al., "Micro expression recognition via dual-stream spatiotemporal attention network," Journal of Healthcare Engineering, Vol. 2021, No. 1, 2021.

[45] G. Zhu et al., "SKD-TSTSAN: Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition," arXiv preprint arXiv:2406.17538, 2024.