Effect of the sun on the measurement of wheat ear density by deep learning

Ear density in the field, i.e. the number of ears per square metre, is one of the yield components of wheat and therefore a variable of high agronomic interest. Its traditional measurement necessitates laborious human observations in the field or destructive sampling. In the recent years, deep learning based on RGB images has been identified as a low-cost, robust and high-throughput alternative to measure this variable. However, most of the studies were limited to the computer challenge of counting the ears in the images, without aiming to convert those counts into ear density. The aim of this study was not only to propose a method for automatic measurement of ear density, but also to evaluate the potential impact of the sun on the measurement.
A same zone of a wheat plot has been imaged by two nadir RGB cameras all over the daily course of the sun, this repeated at flowering, watery ripe, medium milk and hard dough development stages. The bounding boxes of the ears in the images were detected using the YOLOv5 deep learning model, trained on rich existing wheat ear datasets. The shifts between the same elements observed in the images from the two cameras were exploited to compute the image footprint by stereovision. The ear count divided by the image footprint yielded the ear density. To investigate the effect of the sun, a solar spectrum was recorded thanks to a spectrometer at the time of each image acquisition.
The F1 scores of ear bounding box detection at flowering, watery ripe, medium milk and hard dough were respectively 0.87, 0.92, 0.92 and 0.85. At watery ripe and medium milk, the measured ear density was robust during the day and between the two dates. At hard dough stage, increases of sunlight irradiance correlated with decreases of the number of ears detected by deep learning, but also with decreases of the number of ears labelled by humans. This demonstrates that, in some conditions, the wheat ear detection performance indicators based on labelled ear may be misleading regarding the capacity of the machine vision to measure the real ear density.