بهبود ساختار الگوریتم یادگیری عمیق در پردازش تصویر باالهام از ماتریس تفکیک پذیری مغز
محورهای موضوعی : انرژی های تجدیدپذیر
زهرا حیدران داروقه امنیه
1
(گروه برق، دانشکده فنی و مهندسی، واحد ساوه، دانشگاه آزاد اسلامی، ساوه، ایران)
سیدمحمد جلال رستگار فاطمی
2
(گروه برق، دانشکده فنی و مهندسی، واحد ساوه، دانشگاه آزاد اسلامی، ساوه، ایران)
مریم رستگارپور
3
(گروه کامپیوتر، دانشکده فنی و ممهندسی، واحد ساوه، دانشگاه آزاد اسلامی، ساوه، ایران)
کلید واژه: پراکندگی, یادگیری عمیق, سیستم کانولوشنی, نرخ بازشناسی, فیلتر دیفرانسیلی گوسی, سطح انرژی شبکه, ماتریس تفکیک پذیری مغز,
چکیده مقاله :
چکیده: الگوریتم های آموزش عمیق در بسیاری از مسائل بازشناسی الگو، نتایجی در سطح انسان و یا بهتر می توانند به ثبت رسانند. اما این نتایج با مکانیسمی متفاوت از مغز انسان به دست آمده است. مدل پیشنهادی در این مقاله یک الگوریتم تقسیم بندی و درون یابی با الهام از مغز انسان را توصیف نموده و بعد از لایه ورودی، لایه‌ی شبکیه اعمال شده است که به پیروی از شبکیه چشم، عمل رمزنگاری بر روی تصویر ورودی را انجام می‌دهد. سپس تصویر ورودی به فضای ثانی انتقال می‌یابد که تلاش برای تغییر ساختار شبکه عمیق با الهام از مسیر بینایی مغز خواهد بود. بازخورد شبکه، نرخ بازشناسی و سطح انرژی شبکه و یا جامعیت شبکه ی آموزش دیده در زیرمجموعه هایی از مجموعه داده کلتک بررسی می گردد. در نمونه های مشابه الگوریتم های آموزش عمیق برای یادگیری نیاز به داده بیشتری در مقایسه با یادگیری انسان دارد. بعلاوه، اختلاف یادگیری عمیق و انسان، تفاوت در بازنمایش اطلاعات است. در یادگیری عمیق وزن ها در جهتی بهبود می یابند تا در یک آزمایش خاص نتیجه بهینه شود ولی در انسان با میلیون ها سال تکامل، مغز انسان به گونه ای تکامل یافته تا بازنمایش بهینه و مؤثر باشد. چالش مورد بررسی دیگر، عمیق تر شدن لایه های یادگیری عمیق است. تعداد این لایه ها نسبت به مغز چندین برابر گشته است و این مسئله منجر به پیچیدگی و صرف انرژی بیشتر می شود. اما در مغز با صرف انرژی کمتر می تواند تشخیص را انجام دهد. بیشینه نرخ بازشناسی مدل پیشنهادی به 93 درصد می رسد و مدل پایه نزدیک به 91 درصد است. همچنین مدل پیشنهادی تنک‌تر و نرخ آتش نورون ها در لایه های ابتدایی کمتر و پایداری بالایی به تغییرات شدت روشنایی داشته، تفکیک پذیری در لایه های مدل بالاتر رفته و توانسته در مواجهه با تصاویر نویزی پاسخ بهتری نشان دهد و افت بازشناسی کمتری را ثبت کند.
Deep learning algorithms achieves some results at human level or even better in pattern recognition problems. Meanwhile they apply a different mechanism other than human brain. This paper describes a human-inspired segmentation and interpolation algorithm, which applies the retinal layer in the proposed model after the input layer. Following this retina, this layer encrypts the input image and transmits the input image to the second space, which try to change deep network structure inspired of the brain's visual path. Network feedback, recognition rate, and network energy level or the comprehensiveness of the trained network examined in subsets of the Caltech data set. In similar examples, deep learning algorithms require more data to learn other than human. In the difference between deep learning and human, there is a difference in the representation of information. In deep learning, weights improve in a way that optimizes the result in a particular experiment, but in millions of years of human evolution, the human brain has evolved optimally and effectively representation. Another point of contention is the deepening of deep learning layers. The number of these layers has multiplied compared to the brain that lead to more complexity and energy expenditure. However, in the brain it can make a diagnosis with less energy. The maximum recognition rate of the proposed model is 93% and the base model is close to 91%. Also, the proposed model is thinner and the rate of fire of neurons in the initial layers is lower and has a high stability to changes in light intensity. The Dissimilarity of the model layers has been higher and it has been able to show a better response in the face of noise images and record less recognition loss.
[1] M. Riesenhuber, T. Poggio, "Hierarchical models of object recognition in cortex", Nature Neuroscience, vol. 2, no. 11, pp. 1019–25, Nov. 1999 (doi: 10.1038/14819).
[2] K. Fukushima, "Artificial vision by multi-layered neural networks: neocognitron and its advances", Neural Network, vol. 37, no. 2013, pp. 103–19, Jan. 2013 (doi: 10.1016/j.neunet.2012.09.016)
[3] Y. Li, W. Wu, B. Zhang, F. Li, "Enhanced HMAX model with feedforward feature learning for multiclass categorization", Frontiers in Computational Neuroscience, vol. 9, p. 123, Oct. 2015 (doi: 10.3389/fncom.2015.00123).
[4] B. Yang, L. Zhou, Z. Deng, "C-HMAX: Artificial cognitive model inspired by the color vision mechanism of the human brain", Tsinghua Science Technology, vol. 18, no. 1, pp. 51–56, Feb. 2013 (doi: 10.1109/TST.2013.6449407).
[5] S. Zabbah, K. Rajaei, A. Mirzaei, R. Ebrahimpour, S. M. Khaligh-Razavi, "The impact of the lateral geniculate nucleus and corticogeniculate interactions on efficient coding and higher-order visual object processing", Vision Research, vol. 101, pp. 82–93, Aug. 2014 (doi: 10.1016/j.visres.2014.05.006).
[6] A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks", in Advances in neural information processing systems, pp. 1097–1105, 2012.
[7] D. Ciregan, U. Meier, J. Schmidhuber, "Multi-column deep neural networks for image classification", Proceeding of the IEEE/CVPR, pp. 3642–3649, Providence, RI, USA, June 2012 (doi: 10.1109/CVPR.2012.6248110).
[8] D. Berman, A. Buczak, J. Chavis, C. Corbett, "A survey of deep learning methods for cyber security", Information, vol. 10, no. 4, Article Number: 122, April 2019 (doi:10.3390/info10040122).
[9] A. Gupta, H. K. Thakur, R. Shrivastava, P. Kumar, S. Nag, "A big data analysis framework using apache spark and deep learning", Proceeding of the IEEE/ICDMW, pp. 9–16, New Orleans, LA, USA, Nov. 2017 (doi:10.1109/ICDMW.2017.9)
[10] L. Deng, "A tutorial survey of architectures, algorithms, and applications for deep learning", APSIPA Trans. Signal Inf. Process., vol. 3, 2014. (doi: 10.1017/atsip.2013.9)
[11] S. Helie, F. G. Ashby, "Learning and transfer of category knowledge in an indirect categorization task", Psychological Research, vol. 76, no. 3, pp. 292–303, 2012 (doi: 10.1007/s00426-011-0348-1).
[12] M. Stettler, G. Francis, "Using a model of human visual perception to improve deep learning", Neural Networks, vol. 104, pp. 40–49, 2018 (doi: 10.1016/j.neunet.2018.04.005)
[13] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition", Proceedings of the IEEE/CVPR, pp. 770–778, Las Vegas, NV, USA, June 2016 (doi: 10.1109/CVPR.2016.90)
[14] R. C. Fong, W. J. Scheirer, D. D. Cox, "Using human brain activity to guide machine learning", Scientific Reports, vol. 8, no. 1,Article Number: 5397, 2018 (doi: 10.1038/s41598-018-23618-6)
[15] H. Mahdavi-Nasab, M. S. Memarzadeh, P. Moallem, "Automatic Persian license plate recognition by edge detection using hopfield neural network", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 4, pp. 41-48, Winter 2011.
[16] S. Chatterjee, G. D. Field, G. D. Horwitz, E. N. Johnson, K. Koida, K. Mancuso, "Advances in color science: from retina to behavior", Journal Neuroscience, vol. 30, no. 45, pp. 14955–63, Nov. 2010 (doi: 10.1523/JNEUROSCI.4348-10.2010).
[17] M. E. Rudd, “Edge integration in achromatic color perception and the lightness– darkness asymmetry through retinex theory", Journal of Vision, vol. 13, pp. 1–30, Dec. 2013 (doi: 10.1167/13.14.18)
[18] A. Lu, G. Xu, H. Jin, L. Mo, J. Zhang, J. X. Zhang, "Electrophysiological evidence for effects of color knowledge in object recognition", Neuroscience Letters, vol. 469, no. 3, pp. 405–10, Jan. 2010 (doi: 10.1016/j.neulet.2009.12.039).
[19] R. Kozik, M. Chora´s, M. Ficco, F. Palmieri, "A scalable distributed machine learning approach for attack detection in edge computing environments", Journal of Parallel and Distributed Computing, vol. 119, pp. 18 – 26, Sept. 2018 (doi: 10.1016/j.jpdc.2018.03.006).
[20] W. Fu, M. Johnston, M. Zhang, "Automatic construction of Gaussian-based edge detectors using genetic programmings", Applications of Evolutionary Computation, pp. 336-375, 2011 (doi: 10.1016/j.patrec.2005.07.024).
[21] Nili, Hamed, et al. "A toolbox for representational similarity analysis", PLoS Computational Biology, vol. 10, no. 4, April 2014 (doi: 10.1371/journal.pcbi.1003553).
[22] A. R. Karimian, M. Torabian, M. R. Yazdchi, "Improvement of industrial radiography for defect detection of oil and gas pipelines in weld regions by image processing", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 2, pp. 23-30, Summer 2010.
[23] I. F. Salazar-reque, S. G. Huamán, G. Kemper, J. Telles, D. Diaz, "An algorithm for plant disease visual symptom detection in digital images based on superpixels", International Journal on Advanced Science, Engineering and Information Technology, vol. 9, no. 1, pp. 194–203, 2019 (doi: 10.18517/ijaseit.9.1.5322)
[24] J. G. A. Barbedo, "A new automatic method for disease symptom segmentation in digital photographs of plant leaves", European Journal of Plant Pathology, vol. 147, no. 2, pp. 349–364, 2017 (doi: 10.1007/s10658-016-1007-6).
[25] G. Marcus, “Deep Learning: A Critical Appraisal”, arXiv Prepr. arXiv:1801.00631, 2018.
[26] J. Ngiam, Z. Chen, P. W. Koh, A. Y. Ng, "Learning deep energy models", Proceedings of the ICML pp. 1105–1112, June 2011.
[27] K. He, X. Zhang, S. Ren, J. Sun, "Identity mappings in deep residual networks", Computer Vision and Pattern Recognition, pp. 630–645, March 2016.
[28] A. Ng, "Sparse autoencoder", CS294A Lecture Notes, vol. 72, no. 2011, pp. 1–19, 2011.
[29] M. S. Goli, A. Nsghsh, "Robust digital image watermarking against cropping and salt & pepper noise using two-step sudoku", Journal of Intelligent Procedures in Electrical Technology, vol. 8, no. 31, pp. 21-32, Autumn 2017.
_||_[1] M. Riesenhuber, T. Poggio, "Hierarchical models of object recognition in cortex", Nature Neuroscience, vol. 2, no. 11, pp. 1019–25, Nov. 1999 (doi: 10.1038/14819).
[2] K. Fukushima, "Artificial vision by multi-layered neural networks: neocognitron and its advances", Neural Network, vol. 37, no. 2013, pp. 103–19, Jan. 2013 (doi: 10.1016/j.neunet.2012.09.016)
[3] Y. Li, W. Wu, B. Zhang, F. Li, "Enhanced HMAX model with feedforward feature learning for multiclass categorization", Frontiers in Computational Neuroscience, vol. 9, p. 123, Oct. 2015 (doi: 10.3389/fncom.2015.00123).
[4] B. Yang, L. Zhou, Z. Deng, "C-HMAX: Artificial cognitive model inspired by the color vision mechanism of the human brain", Tsinghua Science Technology, vol. 18, no. 1, pp. 51–56, Feb. 2013 (doi: 10.1109/TST.2013.6449407).
[5] S. Zabbah, K. Rajaei, A. Mirzaei, R. Ebrahimpour, S. M. Khaligh-Razavi, "The impact of the lateral geniculate nucleus and corticogeniculate interactions on efficient coding and higher-order visual object processing", Vision Research, vol. 101, pp. 82–93, Aug. 2014 (doi: 10.1016/j.visres.2014.05.006).
[6] A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks", in Advances in neural information processing systems, pp. 1097–1105, 2012.
[7] D. Ciregan, U. Meier, J. Schmidhuber, "Multi-column deep neural networks for image classification", Proceeding of the IEEE/CVPR, pp. 3642–3649, Providence, RI, USA, June 2012 (doi: 10.1109/CVPR.2012.6248110).
[8] D. Berman, A. Buczak, J. Chavis, C. Corbett, "A survey of deep learning methods for cyber security", Information, vol. 10, no. 4, Article Number: 122, April 2019 (doi:10.3390/info10040122).
[9] A. Gupta, H. K. Thakur, R. Shrivastava, P. Kumar, S. Nag, "A big data analysis framework using apache spark and deep learning", Proceeding of the IEEE/ICDMW, pp. 9–16, New Orleans, LA, USA, Nov. 2017 (doi:10.1109/ICDMW.2017.9)
[10] L. Deng, "A tutorial survey of architectures, algorithms, and applications for deep learning", APSIPA Trans. Signal Inf. Process., vol. 3, 2014. (doi: 10.1017/atsip.2013.9)
[11] S. Helie, F. G. Ashby, "Learning and transfer of category knowledge in an indirect categorization task", Psychological Research, vol. 76, no. 3, pp. 292–303, 2012 (doi: 10.1007/s00426-011-0348-1).
[12] M. Stettler, G. Francis, "Using a model of human visual perception to improve deep learning", Neural Networks, vol. 104, pp. 40–49, 2018 (doi: 10.1016/j.neunet.2018.04.005)
[13] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition", Proceedings of the IEEE/CVPR, pp. 770–778, Las Vegas, NV, USA, June 2016 (doi: 10.1109/CVPR.2016.90)
[14] R. C. Fong, W. J. Scheirer, D. D. Cox, "Using human brain activity to guide machine learning", Scientific Reports, vol. 8, no. 1,Article Number: 5397, 2018 (doi: 10.1038/s41598-018-23618-6)
[15] H. Mahdavi-Nasab, M. S. Memarzadeh, P. Moallem, "Automatic Persian license plate recognition by edge detection using hopfield neural network", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 4, pp. 41-48, Winter 2011.
[16] S. Chatterjee, G. D. Field, G. D. Horwitz, E. N. Johnson, K. Koida, K. Mancuso, "Advances in color science: from retina to behavior", Journal Neuroscience, vol. 30, no. 45, pp. 14955–63, Nov. 2010 (doi: 10.1523/JNEUROSCI.4348-10.2010).
[17] M. E. Rudd, “Edge integration in achromatic color perception and the lightness– darkness asymmetry through retinex theory", Journal of Vision, vol. 13, pp. 1–30, Dec. 2013 (doi: 10.1167/13.14.18)
[18] A. Lu, G. Xu, H. Jin, L. Mo, J. Zhang, J. X. Zhang, "Electrophysiological evidence for effects of color knowledge in object recognition", Neuroscience Letters, vol. 469, no. 3, pp. 405–10, Jan. 2010 (doi: 10.1016/j.neulet.2009.12.039).
[19] R. Kozik, M. Chora´s, M. Ficco, F. Palmieri, "A scalable distributed machine learning approach for attack detection in edge computing environments", Journal of Parallel and Distributed Computing, vol. 119, pp. 18 – 26, Sept. 2018 (doi: 10.1016/j.jpdc.2018.03.006).
[20] W. Fu, M. Johnston, M. Zhang, "Automatic construction of Gaussian-based edge detectors using genetic programmings", Applications of Evolutionary Computation, pp. 336-375, 2011 (doi: 10.1016/j.patrec.2005.07.024).
[21] Nili, Hamed, et al. "A toolbox for representational similarity analysis", PLoS Computational Biology, vol. 10, no. 4, April 2014 (doi: 10.1371/journal.pcbi.1003553).
[22] A. R. Karimian, M. Torabian, M. R. Yazdchi, "Improvement of industrial radiography for defect detection of oil and gas pipelines in weld regions by image processing", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 2, pp. 23-30, Summer 2010.
[23] I. F. Salazar-reque, S. G. Huamán, G. Kemper, J. Telles, D. Diaz, "An algorithm for plant disease visual symptom detection in digital images based on superpixels", International Journal on Advanced Science, Engineering and Information Technology, vol. 9, no. 1, pp. 194–203, 2019 (doi: 10.18517/ijaseit.9.1.5322)
[24] J. G. A. Barbedo, "A new automatic method for disease symptom segmentation in digital photographs of plant leaves", European Journal of Plant Pathology, vol. 147, no. 2, pp. 349–364, 2017 (doi: 10.1007/s10658-016-1007-6).
[25] G. Marcus, “Deep Learning: A Critical Appraisal”, arXiv Prepr. arXiv:1801.00631, 2018.
[26] J. Ngiam, Z. Chen, P. W. Koh, A. Y. Ng, "Learning deep energy models", Proceedings of the ICML pp. 1105–1112, June 2011.
[27] K. He, X. Zhang, S. Ren, J. Sun, "Identity mappings in deep residual networks", Computer Vision and Pattern Recognition, pp. 630–645, March 2016.
[28] A. Ng, "Sparse autoencoder", CS294A Lecture Notes, vol. 72, no. 2011, pp. 1–19, 2011.
[29] M. S. Goli, A. Nsghsh, "Robust digital image watermarking against cropping and salt & pepper noise using two-step sudoku", Journal of Intelligent Procedures in Electrical Technology, vol. 8, no. 31, pp. 21-32, Autumn 2017.
