Recombining Features of Frequency Domain and Location for Machine Recognition of Sign Language
Subject Areas : Telecommunication Systems
Sirous Tannaz
1
(Department of Electrical Engineering, Urmia Branch, Islamic Azad University, Urmia, Iran
Microwave and Antenna Research center, Urmia Branch, Islamic Azad University, Urmia, Iran)
Keywords: Sign Language, image segmentation, Radon, Feature extraction,
Abstract :
In this article, a system for recognizing Persian sign language alphabets is presented. This system is able to recognize 32 hand postures for Persian alphabets and translate it into Persian text. For this purpose, images of hand positions have been considered for each letter of the alphabet. The database contains 600 images of different people taken by a digital camera. We have transferred all the image data to the binary domain and resized them with a single scale. Image data preprocessing includes image cropping and noise removal. After pre-processing, 3 algorithms are proposed to extract features. The proposed algorithms include the image segmentation algorithm, the distance between border contour points and the center of gravity algorithm, and Radon transformation. Algorithm of the distances between the border contour points and the center of gravity shows how the points are placed on the peripheral curve of the hand in relation to each other and to the center of gravity, and therefore provides suitable structural information for describing states. The next algorithm is based on image segmentation. In this algorithm, the ratio of the number of white pixels to the total number of pixels is calculated in each of the areas. In Radon transformation, in addition to obtaining the general information of the image in each of the modes, we have increased the accuracy of the detection by using the proposed method and discarding additional information. The proposed methods also provided good results on other image databases.
[1] S. Tannaz and T. Sedghi., “Image Retrieval Using Dynamic Weighting of Compressed High Level Features Framework with LER Matrix ,” Iranian Journal of Electrical & Electronic Engineering, vol. 14, no. 2, pp. 153-161, June 2018 , doi:10.22068/IJEEE.14.2.153.
[2] H. S. Anupama, B. A. Usha, S. Madhushankar, V. Vivek and Y. Kulkarni, "Automated Sign Language Interpreter Using Data Gloves," International Conference on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India, 2021, pp. 472-476, doi: 10.1109/ICAIS50930.2021.9395749.
[3] M. Jalali and T.Sedghi , “Extraction of Multiple Hybrid Features to Reduce the Semantic Vacuum with the Semi-Supervised Classification,” Journal of Communication Engineering Islamic Azad University, vol. 12, no. 45, pp. 153-161, 2022 , doi:10.30495/jce.2022.691134.
[4] X. Zhang, J. Zhao and J. Tian, “A Robust Coinversion Model for Soil Moisture Retrieval From Multisensory Data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 8, pp. 5230-5237, July 2014, doi: 10.1109/tgrs.2013.2287513.
[5] W. Chen,” Single-Shot Imaging Without Reference Wave Using Binary Intensity Pattern for Optically-Secured-Based Correlation, “ IEEE Photonics Journal, vol. 8, no. 1, February 2016, pp. 1943-0655, doi: 10.1109/jphot.2016.2523245.
[6] S. Shafei, H. Vahdati, T.Sedghi and A. Charmin “Modulation Based Combination of High Level Features Generated from SCCF & Contourlet Transforms for CBIR Applications,” Wireless Personal Communications, vol.126, no. 1, pp. 197-208, 05 May 2022, doi: 10.1007/s11277-022-09740-9.
[7] S. Shafei, H. Vahdati, T. Sedghi and A. Charmin , “CBMIR System Based on Matrix Weighting Framework and Linear Transformation with KNN,” Electrica , vol. 22, no. 2, pp. 258-266J , 2022, doi: 10.54614/electrica.2022.210130.
[8] S. Shafei, H. Vahdati, T. Sedghi and A. Charmin , “Novel high level retrieval system based on mathematic algorithm & technique for MRI medical imaging and classification, ” Journal of Instrumentation , vol. 16, no. 17, p. p07055 28, July 2021, doi: 10.1088/1748-0221/16/07/P07055.
[9] M .Jalali, “High-Scale Image Clustering with Semantic Cues Modeling and Spatial Simulation, ” Journal of Southern Communication Engineering Islamic Azad University Bushehr Branch, vol. 12, no. 47,pp. 61-70, Spring 2023, doi :10.30495/jce.2022.1968473.1173.
[10] T. Sedghi, Y. Zeforoosh and M. Jalali, “Response Vector for Calculation of Training Signal based on Progressive Non-Recursive Fusion of Multi-Spectral Image, ” International Journal of Engineering & Technology, vol. 2, no. 1, pp. 30-34, 28 October 2013.
[11] Z. Yang, Y. Ke, T. Chen, M. Grzegorzek and J. See, “Doing More With Moiré Pattern Detection in Digital Photos, ” IEEE Transactions on Image Processing, vol. 32, pp. 694-708, 2023, doi : 10.1109/TIP.2022.3232232.
[12] X. Zhang, S. Zhou, J. Fang and Y. Ni, “Pattern Recognition of Construction Bidding System Based on Image Processing,” International Journal of Computer Systems Science & Engineering, vol. 35, no. 4, pp. 247–256, july 2020, doi : 10.32604/csse.2020.35.247.
[13] A. Shariq, A. Khan, A. M. Khan, M. Khurram, M. F. Umer and M. S. Salam, “Image Processing Based Pattern Recognition and Computerized Embroidery Machine,” Pakistan Journal of Engineering and Technology, PakJET, vol. 5, no. 4, pp. 68–74, 2022, doi :10.51846/vol5iss4pp68-74.
[14] M. Fakheri, T. Sedghi, M. G. Shayesteh and M. C. Amirani, “Framework for image retrieval using machine learnin g and statistical similarity matching techniques, ” IET Image Processingn, vol. 7, no. 1, pp. 1-11, February 2013, doi: 10.1049/iet-ipr.2012.0104.
[15] J. Bao, B. Wang, X. Yang, and H. Zhu, ‘‘Nearest neighbor query in road networks,’’ (in Chinese), Ruan Jian Xue Bao/J. Softw, vol. 29, no. 3, pp. 642–662, Mar. 2018.
[16] H. Li, B. Cai, S. Qiao, Q. Wang, and Y. Wang, “Expanding Tree-Based Continuous K Nearest Neighbor Query in Road Networks With Traffic Rules, ” IEEE Access, vol. 6, pp. 72594–72608, 2018, doi: 10.1109/ACCESS.2018.2881414.
[17] K. Bok, Y. Park and J. Yoo, “An efficient continuous k-nearest neighbor query processing scheme for multimedia data sharing and transmission in location based services ”, Multimedia Tools and Applications, vol. 78, pp. 5403–5426, 2019, doi :10.1007/s11042-018-6433-3.
[18] R. Salman, “Novel Technique in Content Based Image Retrieval using Classification by Deep Learning in Artificial Intelligence,” International Journal of Intelligent Systems and Applications in Engineering, vol. 10, no. 2s, pp. 256-259, Desember 2019.
[19] Gh. Raghuwanshi and V. Tyagi, “Novel Technique for Object Based Image Retrieval Using EM Segmentation for localized image retrieval,” Multimedia Tools and Aplications, vol. 76, no. 12, pp. 13741–13759, June 2017, doi :10.1007/s11042-016-3747-x.
_||_[1] S. Tannaz and T. Sedghi., “Image Retrieval Using Dynamic Weighting of Compressed High Level Features Framework with LER Matrix ,” Iranian Journal of Electrical & Electronic Engineering, vol. 14, no. 2, pp. 153-161, June 2018 , doi:10.22068/IJEEE.14.2.153.
[2] H. S. Anupama, B. A. Usha, S. Madhushankar, V. Vivek and Y. Kulkarni, "Automated Sign Language Interpreter Using Data Gloves," International Conference on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India, 2021, pp. 472-476, doi: 10.1109/ICAIS50930.2021.9395749.
[3] M. Jalali and T.Sedghi , “Extraction of Multiple Hybrid Features to Reduce the Semantic Vacuum with the Semi-Supervised Classification,” Journal of Communication Engineering Islamic Azad University, vol. 12, no. 45, pp. 153-161, 2022 , doi:10.30495/jce.2022.691134.
[4] X. Zhang, J. Zhao and J. Tian, “A Robust Coinversion Model for Soil Moisture Retrieval From Multisensory Data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 8, pp. 5230-5237, July 2014, doi: 10.1109/tgrs.2013.2287513.
[5] W. Chen,” Single-Shot Imaging Without Reference Wave Using Binary Intensity Pattern for Optically-Secured-Based Correlation, “ IEEE Photonics Journal, vol. 8, no. 1, February 2016, pp. 1943-0655, doi: 10.1109/jphot.2016.2523245.
[6] S. Shafei, H. Vahdati, T.Sedghi and A. Charmin “Modulation Based Combination of High Level Features Generated from SCCF & Contourlet Transforms for CBIR Applications,” Wireless Personal Communications, vol.126, no. 1, pp. 197-208, 05 May 2022, doi: 10.1007/s11277-022-09740-9.
[7] S. Shafei, H. Vahdati, T. Sedghi and A. Charmin , “CBMIR System Based on Matrix Weighting Framework and Linear Transformation with KNN,” Electrica , vol. 22, no. 2, pp. 258-266J , 2022, doi: 10.54614/electrica.2022.210130.
[8] S. Shafei, H. Vahdati, T. Sedghi and A. Charmin , “Novel high level retrieval system based on mathematic algorithm & technique for MRI medical imaging and classification, ” Journal of Instrumentation , vol. 16, no. 17, p. p07055 28, July 2021, doi: 10.1088/1748-0221/16/07/P07055.
[9] M .Jalali, “High-Scale Image Clustering with Semantic Cues Modeling and Spatial Simulation, ” Journal of Southern Communication Engineering Islamic Azad University Bushehr Branch, vol. 12, no. 47,pp. 61-70, Spring 2023, doi :10.30495/jce.2022.1968473.1173.
[10] T. Sedghi, Y. Zeforoosh and M. Jalali, “Response Vector for Calculation of Training Signal based on Progressive Non-Recursive Fusion of Multi-Spectral Image, ” International Journal of Engineering & Technology, vol. 2, no. 1, pp. 30-34, 28 October 2013.
[11] Z. Yang, Y. Ke, T. Chen, M. Grzegorzek and J. See, “Doing More With Moiré Pattern Detection in Digital Photos, ” IEEE Transactions on Image Processing, vol. 32, pp. 694-708, 2023, doi : 10.1109/TIP.2022.3232232.
[12] X. Zhang, S. Zhou, J. Fang and Y. Ni, “Pattern Recognition of Construction Bidding System Based on Image Processing,” International Journal of Computer Systems Science & Engineering, vol. 35, no. 4, pp. 247–256, july 2020, doi : 10.32604/csse.2020.35.247.
[13] A. Shariq, A. Khan, A. M. Khan, M. Khurram, M. F. Umer and M. S. Salam, “Image Processing Based Pattern Recognition and Computerized Embroidery Machine,” Pakistan Journal of Engineering and Technology, PakJET, vol. 5, no. 4, pp. 68–74, 2022, doi :10.51846/vol5iss4pp68-74.
[14] M. Fakheri, T. Sedghi, M. G. Shayesteh and M. C. Amirani, “Framework for image retrieval using machine learnin g and statistical similarity matching techniques, ” IET Image Processingn, vol. 7, no. 1, pp. 1-11, February 2013, doi: 10.1049/iet-ipr.2012.0104.
[15] J. Bao, B. Wang, X. Yang, and H. Zhu, ‘‘Nearest neighbor query in road networks,’’ (in Chinese), Ruan Jian Xue Bao/J. Softw, vol. 29, no. 3, pp. 642–662, Mar. 2018.
[16] H. Li, B. Cai, S. Qiao, Q. Wang, and Y. Wang, “Expanding Tree-Based Continuous K Nearest Neighbor Query in Road Networks With Traffic Rules, ” IEEE Access, vol. 6, pp. 72594–72608, 2018, doi: 10.1109/ACCESS.2018.2881414.
[17] K. Bok, Y. Park and J. Yoo, “An efficient continuous k-nearest neighbor query processing scheme for multimedia data sharing and transmission in location based services ”, Multimedia Tools and Applications, vol. 78, pp. 5403–5426, 2019, doi :10.1007/s11042-018-6433-3.
[18] R. Salman, “Novel Technique in Content Based Image Retrieval using Classification by Deep Learning in Artificial Intelligence,” International Journal of Intelligent Systems and Applications in Engineering, vol. 10, no. 2s, pp. 256-259, Desember 2019.
[19] Gh. Raghuwanshi and V. Tyagi, “Novel Technique for Object Based Image Retrieval Using EM Segmentation for localized image retrieval,” Multimedia Tools and Aplications, vol. 76, no. 12, pp. 13741–13759, June 2017, doi :10.1007/s11042-016-3747-x.
