Application and Assessment of Detecto’s Deep Learning Algorithm for Identifying Apple Fruit on Trees

Subject Areas : Sustainable production technologies

Iman Ahmadi ¹ , Fateme Tavassoli ²

1 - Associate Professor, Department of Mechanical Engineering of Agricultural Machinery, Khorasgan Branch, Islamic Azad University, Isfahan, Iran
2 - Master's Degree in Applied Mathematics, Yazd University, Yazd, Iran

Received: 2025-02-01 Accepted : 2025-03-16 Published : 2025-03-20

Keywords: Apple Recognition, Deep Learning, Detecto Algorithm, IOU Criterion,

Abstract :

One area where deep learning models have shown their potential is the identification of objects within images. The focus of this study is to develop a deep learning model to identify apple fruit on a tree using images captured from trees of apple orchards. The performance of the model was evaluated by calculating the ratio of the intersection to the union areas of ​​the actual and predicted boxes surrounding the apples in the image. This was done on a number of close-up images with one or two apples, and then the sensitivity, accuracy, and F1 score metrics were calculated. In order to use all images of the validation folder in evaluating the performance of the algorithm, the average accuracy criterion for apple detection in all images was used, taking into account a detection threshold level of 50%. According to the results acquired, the model training process decreased the cost function's value, and after more than 10 epochs, the graph appeared nearly flat. Furthermore, the variety of apple and the type of the image backgrounds did not influence the model's results. Additionally, regarding quality, the model performed exceptionally well in identifying apples in close-up photos. Quantitatively, the sensitivity, accuracy, and F1 score measures of the model on a number of close-up images with one or two apples were obtained as 1, 0.92, and 0.94, respectively, considering a 80% detection threshold level.. On the other hand, the average accuracy obtained on all images in the validation folder was 88%, considering a 50% detection threshold level. Thus, the effectiveness of the model used in this study was validated in identifying apple fruit on trees.

References:

Berenstein, R., Shahar, O.B., Shapiro, A., Edan, Y., 2010. Grape clusters and foliage detection algorithms for autonomous selective vineyard sprayer. Intel. Serv. Robot. 3, 233–243.
Bi, A., 2022. Build fully-functioning computer vision models with PyTorch. https://github.com/alankbi/detecto.
Chang, A., Yeom, J., Jung, J., Landivar, J., 2020. Comparison of canopy shape and vegetation indices of citrus trees derived from UAV multispectral images for characterization of citrus greening disease. Remote Sensing 12, 4122. https://doi. org/10.3390/rs12244122.
Costa, C., Schurr, U., Loreto, F., Menesatti, P., Carpentier, S., 2019. Plant phenotyping research trends, a science mapping approach. Front. Plant Sci. 9, 1933.
Farjon, G., Krikeb, O., Hillel, A.B., Alchanatis, V., 2019. Detection and counting of flowers on apple trees for better chemical thinning decisions. Precision Agric. 1–19.
Fuentes, A., Yoon, S., Kim, S.C., Park, D.S., 2017. A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors 17, 2022.
Ge, Y., Lin, S., Zhang, Y., Li, Z., Cheng, H., Dong, J., Shao, S., Zhang, J., Qi, X., Wu, Z., 2022. Tracking and counting of tomato at different growth period using an improving YOLO-deepsort network for inspection robot. Machines 10, 489. https:// doi.org/10.3390/machines10060489.
Guti´errez, S., Diago, M.P., Fern´andez-Novales, J., Tardaguila, J., 2018. Vineyard water status assessment using on-the-go thermal imaging and machine learning. PLoS One 13, e0192037.
Magalh˜aes, S.C., dos Santos, F.N., Machado, P., Moreira, A.P., Dias, J., 2023. Benchmarking edge computing devices for grape bunches and trunks detection using accelerated object detection single shot multibox deep learning models. Engineering Applications of Artificial Intelligence 117, 105604. https://doi.org/10.1016/j. engappai.2022.105604.
Marinoudi, V., Sørensen, C.G., Pearson, S., Bochtis, D., 2019. Robotics and labour in agriculture. A context consideration. Biosystem Engineering 184, 111–121. https:// doi.org/10.1016/j.biosystemseng.2019.06.013.
Martínez-Pe˜na, R., V´elez, S., Vacas, R., Martín, H., ´Alvarez, S., 2023. Remote sensing for sustainable pistachio cultivation and improved quality traits evaluation through thermal and non-thermal UAV vegetation indices. Applied Sciences 13, 7716. https://doi.org/10.3390/app13137716.
Padilla, R., S. Netto S.L., da Silva, E.A.B., 2020. A Survey on Performance Metrics for Object-Detection Algorithms. International Conference on Systems, Signals and Image Processing (IWSSIP), Niteroi, Brazil, 2020, pp. 237-242, doi: 10.1109/IWSSIP48289.2020.9145130.
Rong, J., Zhou, H., Zhang, F., Yuan, T., Wang, P., 2023. Tomato cluster detection and counting using improved YOLOv5 based on RGB-D fusion. Computers and Electronics in Agriculture 207, 107741. https://doi.org/10.1016/j. compag.2023.107741.
Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Perez, T., McCool, C., 2016. Deepfruits: A fruit detection system using deep neural networks. Sensors 16, 1222.
Santos, T.T., de Souza, L.L., dos Santos, A.A., Avila, S., 2020. Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association. Comput. Electron. Agric. 170, 105247.
Seo, D., Cho, B.-H., Kim, K.-C., 2021. Development of monitoring robot system for tomato fruits in hydroponic greenhouses. Agronomy 11, 2211. https://doi.org/ 10.3390/agronomy11112211.
Sishodia, R.P., Ray, R.L., Singh, S.K., 2020. Applications of remote sensing in precision agriculture: A review. Remote Sensing 12, 3136. https://doi.org/10.3390/rs12193136.
Tsouros, D.C., Bibi, S., Sarigiannidis, P.G., 2019. A review on UAV-based applications for precision agriculture. Information 10, 349. https://doi.org/10.3390/info10110349.
V´elez, S., Ariza-Sentís, M., Valente, J., 2023. Mapping the spatial variability of Botrytis bunch rot risk in vineyards using UAV multispectral imagery. European Journal of Agronomy 142, 126691. https://doi.org/10.1016/j.eja.2022.126691.
V´elez, S., Rançon, F., Barajas, E., Brunel, G., Rubio, J.A., Tisseyre, B., 2022. Potential of functional analysis applied to Sentinel-2 time-series to assess relevant agronomic parameters at the within-field level in viticulture. Computers and Electronics in Agriculture 194, 106726. https://doi.org/10.1016/j.compag.2022.106726.
Vitzrabin, E., Edan, Y., 2016. Adaptive thresholding with fusion using a rgbd sensor for red sweet-pepper detection. Biosyst. Eng. 146, 45–56.
Wosner, O., Farjon, G., Bar-Hillel, A., 2021. Object detection in agricultural contexts: A multiple resolution benchmark and comparison to human. Computers and Electronics in Agriculture 189, 106404. https://doi.org/10.1016/j. compag.2021.106404.
Xiong, H., Cao, Z., Lu, H., Madec, S., Liu, L., Shen, C., 2019. Tasselnetv2: in-field counting of wheat spikes with context-augmented local regression networks. Plant Methods 15, 150.
Xiong, Y., Ge, Y., From, P.J., 2021. An improved obstacle separation method using deep learning for object detection and tracking in a hybrid visual control loop for fruit picking in clusters. Computers and Electronics in Agriculture 191, 106508. https:// doi.org/10.1016/j.compag.2021.106508.
Xu, G., Khan, A.S., Moshayedi, A.J., Zhang, X., Shuxin, Y., 2022. The object detection, perspective and obstacles in robotic: A review: EAI endorsed trans. AI Robotics 1, e13–e. https://doi.org/10.4108/airo.v1i1.2709.
Zhou, Y., Rui, T., Li, Y., Zuo, X., 2019. A UAV patrol system using panoramic stitching and object detection. Computers and Electrical Engineering 80, 106473. https://doi. org/10.1016/j.compeleceng.2019.106473.