Applying Partial Differential Equations on Cubic Uniform Local Binary Pattern to Reveal Micro-Changes

Esmaeili, V.; Mohassel Feghhi, M.; Shahdi, S. O.

doi:10.22061/jecei.2023.9600.639

Document Type : Original Research Paper

Authors

¹ Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran.

² Department of Electrical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran

https://doi.org/10.22061/jecei.2023.9600.639

Abstract

Background and Objectives: The world we live in everyday, accompany with enormous numbers of minute variations which affect us and our surroundings in several aspects. These variations, so called micro-changes, are low in intensity and brief in duration which makes them almost invisible to naked eyes. Nonetheless, revealing them could open up a new wide range of applications from security, business, engineering, medical, and seismology to psychology.
Methods: In this paper, we adopted a novel autonomous approach comprising Partial Differential Equations (PDE) and Cubic Uniform Local Binary Pattern (Cubic-ULBP) to spot micro-changes. Cubic-ULBP extracts 15 planes containing robust features against noise and illumination variations. Afterwards, PDE pick out single plane out of 15 to reduce time consumption. Results: The proposed method is not only optimized to get the job done perfectly but also provides promising results comparing with most state-of-the-art methods. So that the accuracy is increased about 36% and 40% on the CASME and the CASME II databases, respectively.
Conclusion: The combination of the PDE and the Cubic-ULBP creates a strong and optimal method for detecting the apex frame and micro-movement. This method's performance is found to be promising when the training samples are scarce, too.

Keywords

Main Subjects

Image Processing

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit: http://creativecommons.org/licenses/by/4.0/

Publisher’s Note

JECEI Publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Publisher

Shahid Rajaee Teacher Training University

References

[1] P. Ekman, W. V. Friesen, "Nonverbal leakage and clues to deception," Psychiatry, 32(1): 88-106, 1969.‏

[2] B. Yang, J. Cheng, Y. Yang, B. Zhang, J. Li, "MERTA: micro-expression recognition with ternary attentions," Multimedia Tools Appl., 80(11): 1-16, 2021.

[3] L. Zhou, X. Shao, Q. Mao, "A survey of micro-expression recognition," Image Vision Comput., 105: 104043, 2021.‏

[4] V. Esmaeili, S. O. Shahdi, "Automatic micro-expression apex spotting using Cubic-LBP," Multimedia Tools Appl., 79(27): 20221-20239, 2020.

[5] V. Esmaeili, M. M. Feghhi, S. O. Shahdi, "Micro-Expression recognition using histogram of image gradient orientation on diagonal planes," in Proc. 2021 5th International Conference on Pattern Recognition and Image Analysis (IPRIA): 1-5, 2021.

[6] V. Esmaeili, M. Mohassel Feghhi, S. O. Shahdi, "Autonomous apex detection and micro-expression recognition using proposed diagonal planes," Int. J. Nonlinear Anal. Appl., 11: 483-497,‏ 2020.

[7] V. Esmaeili, M. Mohassel Feghhi, S. O. Shahdi, "A comprehensive survey on facial micro-expression: approaches and databases," Multimedia Tools Appl., 81: 40089- 40134, 2022.

[8] V. Esmaeili, M. Mohassel Feghhi, S. O. Shahdi, "Automatic micro-expression apex frame spotting using local binary pattern from six intersection planes," arXiv preprint arXiv: 2104.02149, 2021.‏

[9] M. Xiao, Y. Chen, J. Zhu, H. Zhang, X. Zhao, L. Gao, W. Xing, "Climbing the apex of the ORR volcano plot via binuclear site construction: electronic and geometric engineering," J. the Am. Chem. Soc., 141(44): 17763-17770,‏ 2019.

[10] K. S. Exner, "Does a thermoneutral electrocatalyst correspond to the apex of a volcano plot for a simple two‐electron process?," Angew. Chem. Int. Ed., 59(26): 10236-10240, 2020.

[11] C. Widiwijayanti, J. Déverchère, R. Louat, M. Sébrier, H. Harjono, M. Diament, D. Hidayat, "Aftershock sequence of the 1994, Mw 6.8, Liwa earthquake (Indonesia): Seismic rupture process in a volcanic arc," Geophys. Res. Lett., 23(21): 3051-3054,‏ 1996.

[12] J. Skinner, "The smoke of an eruption and the dust of an earthquake: Dark tourism, the sublime, and the re-animation of the disaster location," in the Palgrave handbook of dark tourism studies. Palgrave Macmillan, London: 125-150,‏ 2018.

[13] O. Yildirim, M. Talo, E. J. Ciaccio, R. San Tan, U. R. Acharya, "Accurate deep neural network model to detect cardiac arrhythmia on more than 10,000 individual subject ECG records," Comput. Methods Programs Biomed., 197: 105740,‏ 2020.

[14] E. Moghadas, J. Rezazadeh, R. Farahbakhsh, "An IoT patient monitoring based on fog computing and data mining: Cardiac arrhythmia usecase," Internet Things, 11: 100251,‏ 2020.

[15] S. T. Liong, J. See, K. Wong, R. C. W. Phan, "Less is more: Micro-expression recognition from video using apex frame," Signal Process. Image Commun., 62: 82-92,‏ 2018.

[16] Y. Li, X. Huang, G. Zhao, "Can micro-expression be recognized based on single apex frame?," in Proc. 2018 25th IEEE International Conference on Image Processing (ICIP): 3094-3098,‏ 2018.

[17] G. Zhao, M. Pietikainen, "Dynamic texture recognition using local binary patterns with an application to facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., 29(6): 915-928,‏ 2007.

[18] T. Ojala, M. Pietikäinen, D. Harwood, "A comparative study of texture measures with classification based on featured distributions," Pattern Recogn., 29(1): 51-59,‏ 1996.

[19] T. Ojala, M. Pietikainen, T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Trans. Pattern Anal. Mach. Intell., 24(7): 971-987,‏ 2002.

[20] V. Esmaeili, M. Mohassel Feghhi, S. O. Shahdi, "Micro-expression recognition based on the multi-color ulbp and histogram of gradient direction from six intersection planes," J. Iran. Associ. Electr. Electron. Eng., 19(3): 123-130, 2022.

[21] V. Esmaeili, M. Mohassel Feghhi, S. O. Shahdi, "Spotting micro‐movements in image sequence by introducing intelligent cubic‐LBP," IET Image Process., 16(14): 3814-3830, 2022.‏

[22] V. Esmaeili, M. Mohassel Feghhi, S. O. Shahdi, "Early COVID-19 diagnosis from lung ultrasound images combining RIULBP-TP and 3D-DenseNet," in Proc. 2022 9th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS): 1-5, 2022.

[23] V. Esmaeili, M. Mohassel Feghhi, "Real-time authentication for electronic service applicants using a method based on two-stream 3d deep learning," Soft Comput. J., 2023.

[24] V. Esmaeili, M. Mohassel Feghhi, "Diagnosis of Covid-19 disease by combining hand-crafted and deep-learning methods on ultrasound data," J. Mach. Vision Image Process., 9(4): 31-41, 2022.

[25] V. Esmaeili, M. Mohassel Feghhi, "COVID-19 diagnosis: ULBPFP-net approach on lung ultrasound data," Iran. J. Electr. Electron. Eng., 19(3): 2586-2586, 2023.

[26] V. Esmaeili, M. Mohassel Feghhi, S. O. Shahdi, "Automatic micro-expression recognition using LBP-SIPl and FR-CNN," AUT J. Model. Simul., 54(1): 59-72, 2022.

[27] N. Wadhwa, M. Rubinstein, F. Durand, W. T. Freeman, "Phase-based video motion processing," ACM Trans. Graphics (TOG), 32(4): 1-10, 2013.

[28] H. Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, W. Freeman, "Eulerian video magnification for revealing subtle changes in the world," ACM Trans. Graphics (TOG), 31(4): 1-8, 2012.

[29] G. Fahmy, M. F. Fahmy, O. M. Fahmy, "Micro-movement magnification in video signals using complex wavelet analysis," IET Image Process., 11(11): 986-993,‏ 2017.

[30] C. Liu, A. Torralba, W. T. Freeman, F. Durand, E. H. Adelson, "Motion magnification," ACM Trans. Graphics (TOG), 24(3): 519-526,‏ 2005.

[31] M. Janatka, A. Sridhar, J. Kelly, D. Stoyanov, "Higher order of motion magnification for vessel localisation in surgical video," in Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention: 307-314, 2018.

[32] X. Li, X. Hong, A. Moilanen, X. Huang, T. Pfister, G. Zhao, M. Pietikäinen, "Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods," IEEE Trans. Affective Comput., 9(4): 563-577, 2017.

[33] M. Lu, Y. Chai, Q. Liu, "Observation of tower vibration based on subtle motion magnification," IFAC-PapersOnLine, 52(24): 346-350, 2019.

[34] W. Fan, Z. Zheng, W. Zeng, Y. Chen, H. Q. Zeng, H. Shi, X. Luo, "Robotically surgical vessel localization using robust hybrid video motion magnification," IEEE Rob. Autom. Lett., 6(2): 1567-1573, 2021.

[35] A. Mohseni, "A new PDE-based resolution enhancement technique for the analysis of low SNR particle displacement images," Eur. J. Mech. B.Fluids, 85: 289-311,‏ 2021.

[36] T. Barbu, "Feature keypoint-based image compression technique using a well-posed nonlinear fourth-order PDE-based model," Mathematics, 8(6): 930,‏ 2020.

[37] L. Afraites, A. Hadri, A. Laghrib, M. Nachaoui, "A high order PDE-constrained optimization for the image denoising problem," Inverse Probl. Sci. Eng., 29(12):1821-1863, ,‏ 2021.

[38] A. K. Al-Jaberi, E. M. Hameed, "Topological data analysis for evaluating PDE-based denoising models," J. Phys. Conf. Ser., 1897(1): 012006. 2021.

[39] C. Fang, Z. Zhao, P. Zhou, Z. Lin, "Feature learning via partial differential equation with applications to face recognition," Pattern Recogn., 69: 14-25, 2017.

[40] S. Moorthi, S. Karthikeyan, "A Study On Learning Partial Differential Equations Algorithm For Face Recognition," Int. J. Innovative Res. Explorer, 5(4): 127-134, 2018.

[41] W. J. Yan, Q. Wu, Y. J. Liu, S. J. Wang, X. Fu, "CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces," in Proc. 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG): 1-7, 2013.

[42] W. J. Yan, X. Li, S. J. Wang, G. Zhao, Y. J. Liu, Y. H. Chen, X. Fu, "CASME II: An improved spontaneous micro-expression database and the baseline evaluation," PloS one, 9(1): e86041, 2014.

[43] A. J. Witsil, J. B. Johnson, "Volcano video data characterized and classified using computer vision and machine learning algorithms," Geosci. Front., 11(5): 1789-1803,‏ 2020.

[44] W. Fuhl, T. Kübler, K. Sippel, W. Rosenstiel, E. Kasneci, "Excuse: Robust pupil detection in real-world scenarios," in Proc. International Conference on Computer Analysis of Images and Patterns: 39-51, 2015.

[45]C. Nicholson, "Evaluation metrics for machine learning—accuracy, precision, recall, and F1 defined," ed: Pathmind. http://pathmind. com/wiki/accuracy-precision-recall-f1, 2019.

LETTERS TO EDITOR

Journal of Electrical and Computer Engineering Innovations (JECEI) welcomes letters to the editor for the post-publication discussions and corrections which allows debate post publication on its site, through the Letters to Editor. Letters pertaining to manuscript published in JECEI should be sent to the editorial office of JECEI within three months of either online publication or before printed publication, except for critiques of original research. Following points are to be considering before sending the letters (comments) to the editor.

[1] Letters that include statements of statistics, facts, research, or theories should include appropriate references, although more than three are discouraged.

[2] Letters that are personal attacks on an author rather than thoughtful criticism of the author’s ideas will not be considered for publication.

[3] Letters can be no more than 300 words in length.

[4] Letter writers should include a statement at the beginning of the letter stating that it is being submitted either for publication or not.

[5] Anonymous letters will not be considered.

[6] Letter writers must include their city and state of residence or work.

[7] Letters will be edited for clarity and length.

Name *

Email Address *

Affiliation *

Comments *

Security Code *

Journal of Electrical and Computer Engineering Innovations (JECEI)

Applying Partial Differential Equations on Cubic Uniform Local Binary Pattern to Reveal Micro-Changes

References

References

Send comment about this article

Volume 12, Issue 1
January 2024
Pages 259-270

Applying Partial Differential Equations on Cubic Uniform Local Binary Pattern to Reveal Micro-Changes

References

References

Send comment about this article

Volume 12, Issue 1January 2024Pages 259-270

Volume 12, Issue 1
January 2024
Pages 259-270