[en] Traditional wheat head detection and segmentation methods based on machine learning algorithms suffer from issues such as low efficiency and poor accuracy, resulting in the algorithms’ inability to generalize. The recent advances in deep learning, specifically in object detection methods, as well as computer development, have enabled the development of robust wheat head detection and segmentation methods. However, while international datasets of box labels are available for head detection, mask labels for segmentation are missing, and collecting them on a large scale is prohibitively expensive, time-consuming, and difficult. In this paper, we propose an unsupervised approach for segmenting wheat heads based only on box labels. Multiple state-of-the-art object detection methods have been trained on reference datasets and our collected data in order to find the best model to extract head bounding boxes. The obtained boxes were used as input of an unsupervised segmentation model named DeepMAC, which predicts the head mask in each box. Then, those masks are exploited to train several state-of-the-art supervised segmentation models. These models showed promising results on the collected dataset, covering all the wheat development stages. The average F1 score of head bounding box detection is 0.93 and the average F1 score of segmentation is 0.86.
Disciplines :
Electrical & electronics engineering
Author, co-author :
Ennadifi, Elias ; Université de Mons - UMONS
Dandrifosse, Sébastien
Mokhtari, Mohammed El Amine ; Université de Mons - UMONS > Faculté Polytechnique > Service Information, Signal et Intelligence artificielle
Carlier, Alexis
Laraba, Sohaib ; Université de Mons - UMONS > Faculté Polytechnique > Service Information, Signal et Intelligence artificielle
Mercatoris, Benoit
Gosselin, Bernard ; Université de Mons - UMONS > Faculté Polytechnique > Service Information, Signal et Intelligence artificielle
Language :
English
Title :
Local Unsupervised Wheat Head Segmentation
Publication date :
22 September 2022
Event name :
2022 IEEE 18th International Conference on Intelligent Computer Communication and Processing
Event place :
Cluj-Napoca, Romania
Event date :
September 22-24 2022
Event number :
18
By request :
Yes
Audience :
International
Journal title :
ICCP 2022
Peer reviewed :
Peer reviewed
Research unit :
F105 - Information, Signal et Intelligence artificielle
Research institute :
R450 - Institut NUMEDIART pour les Technologies des Arts Numériques
Name of the research project :
3970 - PHENWHEAT - Caractérisation de la dynamique de croissance de cultures de froment d’hiver au moyen d’une plateforme de phénotypage par proxidétection en conditions variables de stress biotique et abiotique - Région wallonne
M. P. Pound, J. A. Atkinson, D. M. Wells, T. P. Pridmore, and A. P. French, "Deep learning for multi-task plant phenotyping," in Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 2055-2063.
M. M. Hasan, J. P. Chopin, H. Laga, and S. J. Miklavcic, "Detection and analysis of wheat spikes using Convolutional Neural Networks," Plant Methods, vol. 14, no. 1, pp. 1-13, 2018. [Online]. Available: https://doi.org/10.1186/s13007-018-0366-8
S. Madec, X. Jin, H. Lu, B. De Solan, S. Liu, F. Duyme, E. Heritier, and F. Baret, "Ear density estimation from high resolution RGB imagery using deep learning technique," Agricultural and Forest Meteorology, vol. 264, no. May 2018, pp. 225-234, 2019. [Online]. Available: https://doi.org/10.1016/j.agrformet.2018.10.013
H. Xiong, Z. Cao, H. Lu, S. Madec, L. Liu, and C. Shen, "TasselNetv2: In-field counting of wheat spikes with context-augmented local regression networks," Plant Methods, vol. 15, no. 1, pp. 0-14, 2019. [Online]. Available: https://doi.org/10.1186/s13007-019-0537-2
Y. Yang, X. Huang, L. Cao, L. Chen, and K. Huang, "Field Wheat Ears Count Based on YOLOv3," in Proceedings - 2019 International Conference on Artificial Intelligence and Advanced Manufacturing, AIAM 2019. Manhattan, NY, USA: IEEE, 2019, pp. 444-448.
L. Cao, X. Zhang, J. Pu, S. Xu, X. Cai, and Z. Li, "The field wheat count based on the efficientdet algorithm," in 2020 IEEE 3rd International Conference on Information Systems and Computer Aided Education (ICISCAE). IEEE, 2020, pp. 557-561.
H. Lu and Z. Cao, "TasselNetV2+: A Fast Implementation for High- Throughput Plant Counting From High-Resolution RGB Imagery," Frontiers in Plant Science, vol. 11, no. December, pp. 1-15, 2020.
X. Xu, H. Li, F. Yin, L. Xi, H. Qiao, Z. Ma, S. Shen, B. Jiang, and X. Ma, "Wheat ear counting using K-means clustering segmentation and convolutional neural network," Plant Methods, vol. 16, no. 1, pp. 1-13, 2020. [Online]. Available: https://doi.org/10.1186/s13007-020-00648-8
J. Zhao, X. Zhang, J. Yan, X. Qiu, X. Yao, Y. Tian, Y. Zhu, and W. Cao, "A Wheat Spike Detection Method in UAV Images Based on Improved YOLOv5," Remote Sensing, vol. 13, pp. 1-16, 2021.
Y. Zhaosheng, L. Tao, Y. Tianle, J. Chengxin, and S. Chengming, "Rapid Detection of Wheat Ears in Orthophotos From Unmanned Aerial Vehicles in Fields Based on YOLOX," Frontiers in Plant Science, vol. 13, no. April, apr 2022. [Online]. Available: https: //www.frontiersin.org/articles/10.3389/fpls.2022.851245/full
E. David, S. Madec, P. Sadeghi-Tehran, H. Aasen, B. Zheng, S. Liu, N. Kirchgessner, G. Ishikawa, K. Nagasawa, M. A. Badhon, C. Pozniak, B. de Solan, A. Hund, S. C. Chapman, F. Baret, I. Stavness, and W. Guo, "Global wheat head detection (GWHD) Dataset: A large and diverse dataset of high-resolution RGB-labelled images to develop and benchmark wheat head detection methods," Plant Phenomics, vol. 2020, 2020.
E. David, M. Serouart, D. Smith, S. Madec, K. Velumani, S. Liu, X. Wang, F. Pinto, S. Shafiee, I. S. A. Tahir, H. Tsujimoto, S. Nasuda, B. Zheng, N. Kirchgessner, H. Aasen, A. Hund, P. Sadhegi-Tehran, K. Nagasawa, G. Ishikawa, S. Dandrifosse, A. Carlier, B. Dumont, B. Mercatoris, B. Evers, K. Kuroki, H. Wang, M. Ishii, M. A. Badhon, C. Pozniak, D. S. LeBauer, M. Lillemo, J. Poland, S. Chapman, B. de Solan, F. Baret, I. Stavness, and W. Guo, "Global Wheat Head Detection 2021: An Improved Dataset for Benchmarking Wheat Head Detection Methods," Plant Phenomics, vol. 2021, no. May, pp. 1-9, sep 2021. [Online]. Available: https://spj.sciencemag.org/journals/plantphenomics/2021/9846158/
T. W. Ayalew, J. R. Ubbens, and I. Stavness, "Unsupervised Domain Adaptation for Plant Organ Counting," Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12540 LNCS, pp. 330-346, 2020.
B. Gong, D. Ergu, Y. Cai, and B. Ma, "Real-Time Detection for Wheat Head Applying Deep Neural Network," Sensors, vol. 21, no. 1, p. 191, dec 2020. [Online]. Available: https://www.mdpi.com/1424-8220/21/1/ 191
Y. Wu, Y. Hu, and L. Li, "BTWD: Bag of Tricks for Wheat Detection," in Lecture Notes in Computer Science. New-York, NY, USA: Springer International Publishing, 2020, vol. 12540 LNCS, no. 2, pp. 450-460. [Online]. Available: http://dx.doi.org/10.1007/978-3-030-65414-6 31
S. Bhagat, M. Kokare, V. Haswani, P. Hambarde, and R. Kamble, "WheatNet-Lite: A Novel Light Weight Network for Wheat Head Detection," Proceedings of the IEEE International Conference on Computer Vision, vol. 2021-Octob, pp. 1332-1341, 2021.
F. Fourati, W. S. Mseddi, and R. Attia, "Wheat Head Detection using Deep, Semi-Supervised and Ensemble Learning," Canadian Journal of Remote Sensing, vol. 0, no. 0, pp. 1-13, 2021. [Online]. Available: https://doi.org/10.1080/07038992.2021.1906213
Z. K. Hartley and A. P. French, "Domain adaptation of synthetic images for wheat head detection," Plants, vol. 10, no. 12, pp. 1-11, 2021.
A. Lad and M. S. Raval, "Resolving Issues with Wheat Head Detection: A Use Case of XAI in Agriculture Scenario," in Lecture Notes in Computer Science. New-York, NY, USA: Springer, 2021, pp. 1-6.
J. Li, C. Li, S. Fei, J. Shi, Z. Xiao, C. Ma, W. Chen, F. Ding, Y. Wang, and Y. Li, "Wheat ear recognition based on retinanet and transfer learning," Sensors, vol. 21, no. 14, 2021.
K. Najafian, A. Ghanbari, I. Stavness, L. Jin, G. Hassan Shirdel, and F. Maleki, "A Semi-self-supervised Learning Approach for Wheat Head Detection using Extremely Small Number of Labeled Samples," Proceedings of the IEEE International Conference on Computer Vision, vol. 2021-Octob, pp. 1342-1351, 2021.
Y. Wang, Y. Qin, and J. Cui, "Occlusion Robust Wheat Ear Counting Algorithm Based on Deep Learning," Frontiers in Plant Science, vol. 12, no. June, pp. 1-14, 2021.
Y. Dong, Y. Liu, H. Kang, C. Li, P. Liu, and Z. Liu, "Lightweight and efficient neural network with SPSA attention for wheat ear detection," PeerJ Computer Science, vol. 8, p. e931, 2022.
S. Khaki, N. Safaei, H. Pham, and L. Wang, "WheatNet: A lightweight convolutional neural network for high-throughput image-based wheat head detection and counting," Neurocomputing, vol. 489, pp. 78-89, 2022. [Online]. Available: https://doi.org/10.1016/j.neucom.2022.03.017
C. Liu, K. Wang, H. Lu, and Z. Cao, "Dynamic Color Transform Networks for Wheat Head Detection," Plant Phenomics, vol. 2022, 2022.
J. Sun, K. Yang, C. Chen, J. Shen, Y. Yang, X. Wu, and T. Norton, "Wheat head counting in the wild by an augmented feature pyramid networks-based convolutional neural network," Computers and Electronics in Agriculture, vol. 193, no. April 2021, p. 106705, 2022. [Online]. Available: https://doi.org/10.1016/j.compag.2022.106705
C. Wen, J. Wu, H. Chen, H. Su, X. Chen, Z. Li, and C. Yang, "Wheat Spike Detection and Counting in the Field Based on SpikeRetinaNet," Frontiers in Plant Science, vol. 13, no. March, pp. 1-16, 2022.
X. Xiong, L. Duan, L. Liu, H. Tu, P. Yang, D. Wu, G. Chen, L. Xiong, W. Yang, and Q. Liu, "Panicle-SEG: A robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization," Plant Methods, vol. 13, no. 1, pp. 1-15, 2017. [Online]. Available: https://doi.org/10.1186/s13007-017-0254-7
J. Ma, Y. Li, K. Du, F. Zheng, L. Zhang, Z. Gong, and W. Jiao, "Segmenting ears of winter wheat at flowering stage using digital images and deep learning," Computers and Electronics in Agriculture, vol. 168, no. November 2019, p. 105159, 2020. [Online]. Available: https://doi.org/10.1016/j.compag.2019.105159
J. Ma, Y. Li, H. Liu, K. Du, F. Zheng, Y. Wu, and L. Zhang, "Improving segmentation accuracy for ears of winter wheat at flowering stage by semantic segmentation," Computers and Electronics in Agriculture, vol. 176, no. July, p. 105662, 2020. [Online]. Available: https://doi.org/10.1016/j.compag.2020.105662
S. Rawat, A. L. Chandra, S. V. Desai, V. N. Balasubramanian, S. Ninomiya, and W. Guo, "How Useful Is Image-Based Active Learning for Plant Organ Segmentation?" Plant Phenomics, vol. 2022, pp. 1-11, feb 2022. [Online]. Available: https://spj.sciencemag.org/ journals/plantphenomics/2022/9795275/
J. Zhang, A. Min, B. J. Steffenson, W.-H. Su, C. D. Hirsch, J. Anderson, J. Wei, Q. Ma, and C. Yang, "Wheat-Net: An Automatic Dense Wheat Spike Segmentation Method Based on an Optimized Hybrid Task Cascade Model," Frontiers in Plant Science, vol. 13, no. February, pp. 1-13, 2022.
Z. Grbović, M. Panić, O. Marko, S. Brdar, and V. Crnojević, "Wheat Ear Detection in RGB and Thermal Images Using Deep Neural Networks," in International Conference on Machine Learning and Data Mining, MLDM 2019. Fockendorf, Germany: ibai, 2019. [Online]. Available: https://www.researchgate.net/publication/336413241
P. Sadeghi-Tehran, N. Virlet, E. M. Ampe, P. Reyns, and M. J. Hawkesford, "DeepCount: In-Field Automatic Quantification of Wheat Spikes Using Simple Linear Iterative Clustering and Deep Convolutional Neural Networks," Frontiers in Plant Science, vol. 10, no. September, pp. 1-16, 2019.
D. Wang, Y. Fu, G. Yang, X. Yang, D. Liang, C. Zhou, N. Zhang, H. Wu, and D. Zhang, "Combined Use of FCN and Harris Corner Detection for Counting Wheat Ears in Field Conditions," IEEE Access, vol. 7, pp. 178 930-178 941, 2019.
W. H. Su, J. Zhang, C. Yang, R. Page, T. Szinyei, C. D. Hirsch, and B. J. Steffenson, "Automatic evaluation of wheat resistance to fusarium head blight using dual mask-rcnn deep learning frameworks in computer vision," Remote Sensing, vol. 13, no. 1, pp. 1-20, 2021.
D. Wang, D. Zhang, G. Yang, B. Xu, Y. Luo, and X. Yang, "SSRNet: In-Field Counting Wheat Ears Using Multi-Stage Convolutional Neural Network," IEEE Transactions on Geoscience and Remote Sensing, vol. 60, 2022.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 580-587.
R. Girshick, "Fast r-cnn," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1440-1448.
S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," Advances in neural information processing systems, vol. 28, 2015.
J. R. Uijlings, K. E. Van De Sande, T. Gevers, and A. W. Smeulders, "Selective search for object recognition," International journal of computer vision, vol. 104, no. 2, pp. 154-171, 2013.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779-788.
J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4: Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "Ssd: Single shot multibox detector," in European conference on computer vision. Springer, 2016, pp. 21-37.
M. Tan, R. Pang, and Q. V. Le, "Efficientdet: Scalable and efficient object detection," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10 781-10 790.
O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234-241.
K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961-2969.
V. Birodkar, Z. Lu, S. Li, V. Rathod, and J. Huang, "The surprising impact of mask-head architecture on novel class segmentation," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 7015-7025.
A. Carlier, S. Dandrifosse, B. Dumont, and B. Mercatoris, "Wheat ear segmentation based on a multisensor system and superpixel classification," Plant Phenomics, vol. 2022, 2022.
