Critical evaluation of the effects of a cross-validation strategy and machine learning optimization on the prediction accuracy and transferability of a soybean yield prediction model using UAV-based remote sensing

Luthfan Nur Habibi; Tsutomu Matsui; Takashi Tanaka

doi:10.1016/j.jafr.2024.101096

Critical evaluation of the effects of a cross-validation strategy and machine learning optimization on the prediction accuracy and transferability of a soybean yield prediction model using UAV-based remote sensing

Luthfan Nur Habibi, Tsutomu Matsui, Takashi Tanaka^*

^*Corresponding author for this work

Department of Agroecology - Crop Health

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaper › Journal article › Research › peer-review

Abstract

Crop yield prediction models are critical tools for evaluating growth performance and informing decisions during farm management. Developing yield prediction models that are robust not only in the ranges of the model spatial domain but also in additional locations using a data-driven approach is challenging. The main objective of this study was to investigate an appropriate cross-validation (CV) strategy for establishing transferable UAV-based yield prediction models across different spatial domains (i.e., meeting extrapolation mapping objectives). In this study, we compared three data splitting procedures for the CV protocols, including random data splitting (random CV), cluster-based spatial splitting (spatial CV), and field-specific hold-out data splitting (leave-one-field-out CV). Model optimization was also examined to determine whether these factors affect the transferability of the yield model, including performing recursive feature elimination (RFE) and comparing the effects of algorithms utilized in the yield prediction model. Three base learner algorithms, namely, random forest, XGBoost, and LASSO regression, were utilized, and a stacked ensemble technique model formed with these base learners was also implemented. The established models were later tested on an independent field as a test dataset to evaluate the model transferability performance. Random CV exhibited poor error tracking performance in predicting yield beyond the model spatial domain, while spatial CV and leave-one-field-out CV approaches provided better expectation on yield predictions outside the model's training spatial domain. Furthermore, simple models as implementing LASSO regression and RFE improved the model capability in extrapolation tasks. The results of this study suggest that spatially-aware CV should be used as the standard method rather than conventional random CV for validating the yield model to ensure a more realistic and reliable yield model in extrapolation objectives.

Original language	English
Article number	101096
Journal	Journal of Agriculture and Food Research
DOIs	https://doi.org/10.1016/j.jafr.2024.101096
Publication status	Published - Mar 2024

Keywords

Spatial data
Leave-one-field-out
Extrapolation
Vegetation indices
Spatial clustering

Access to Document

10.1016/j.jafr.2024.101096Licence: CC BY

Fingerprint

Dive into the research topics of 'Critical evaluation of the effects of a cross-validation strategy and machine learning optimization on the prediction accuracy and transferability of a soybean yield prediction model using UAV-based remote sensing'. Together they form a unique fingerprint.

Cite this

@article{79414d0d75a245f993248e6da8b42260,

title = "Critical evaluation of the effects of a cross-validation strategy and machine learning optimization on the prediction accuracy and transferability of a soybean yield prediction model using UAV-based remote sensing",

abstract = "Crop yield prediction models are critical tools for evaluating growth performance and informing decisions during farm management. Developing yield prediction models that are robust not only in the ranges of the model spatial domain but also in additional locations using a data-driven approach is challenging. The main objective of this study was to investigate an appropriate cross-validation (CV) strategy for establishing transferable UAV-based yield prediction models across different spatial domains (i.e., meeting extrapolation mapping objectives). In this study, we compared three data splitting procedures for the CV protocols, including random data splitting (random CV), cluster-based spatial splitting (spatial CV), and field-specific hold-out data splitting (leave-one-field-out CV). Model optimization was also examined to determine whether these factors affect the transferability of the yield model, including performing recursive feature elimination (RFE) and comparing the effects of algorithms utilized in the yield prediction model. Three base learner algorithms, namely, random forest, XGBoost, and LASSO regression, were utilized, and a stacked ensemble technique model formed with these base learners was also implemented. The established models were later tested on an independent field as a test dataset to evaluate the model transferability performance. Random CV exhibited poor error tracking performance in predicting yield beyond the model spatial domain, while spatial CV and leave-one-field-out CV approaches provided better expectation on yield predictions outside the model's training spatial domain. Furthermore, simple models as implementing LASSO regression and RFE improved the model capability in extrapolation tasks. The results of this study suggest that spatially-aware CV should be used as the standard method rather than conventional random CV for validating the yield model to ensure a more realistic and reliable yield model in extrapolation objectives.",

keywords = "Spatial data, Leave-one-field-out, Extrapolation, Vegetation indices, Spatial clustering",

author = "Habibi, {Luthfan Nur} and Tsutomu Matsui and Takashi Tanaka",

year = "2024",

month = mar,

doi = "10.1016/j.jafr.2024.101096",

language = "English",

journal = "Journal of Agriculture and Food Research",

issn = "2666-1543",

publisher = "Elsevier",

}

Critical evaluation of the effects of a cross-validation strategy and machine learning optimization on the prediction accuracy and transferability of a soybean yield prediction model using UAV-based remote sensing. / Habibi, Luthfan Nur; Matsui, Tsutomu; Tanaka, Takashi.
In: Journal of Agriculture and Food Research, 03.2024.

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaper › Journal article › Research › peer-review

TY - JOUR

T1 - Critical evaluation of the effects of a cross-validation strategy and machine learning optimization on the prediction accuracy and transferability of a soybean yield prediction model using UAV-based remote sensing

AU - Habibi, Luthfan Nur

AU - Matsui, Tsutomu

AU - Tanaka, Takashi

PY - 2024/3

Y1 - 2024/3

N2 - Crop yield prediction models are critical tools for evaluating growth performance and informing decisions during farm management. Developing yield prediction models that are robust not only in the ranges of the model spatial domain but also in additional locations using a data-driven approach is challenging. The main objective of this study was to investigate an appropriate cross-validation (CV) strategy for establishing transferable UAV-based yield prediction models across different spatial domains (i.e., meeting extrapolation mapping objectives). In this study, we compared three data splitting procedures for the CV protocols, including random data splitting (random CV), cluster-based spatial splitting (spatial CV), and field-specific hold-out data splitting (leave-one-field-out CV). Model optimization was also examined to determine whether these factors affect the transferability of the yield model, including performing recursive feature elimination (RFE) and comparing the effects of algorithms utilized in the yield prediction model. Three base learner algorithms, namely, random forest, XGBoost, and LASSO regression, were utilized, and a stacked ensemble technique model formed with these base learners was also implemented. The established models were later tested on an independent field as a test dataset to evaluate the model transferability performance. Random CV exhibited poor error tracking performance in predicting yield beyond the model spatial domain, while spatial CV and leave-one-field-out CV approaches provided better expectation on yield predictions outside the model's training spatial domain. Furthermore, simple models as implementing LASSO regression and RFE improved the model capability in extrapolation tasks. The results of this study suggest that spatially-aware CV should be used as the standard method rather than conventional random CV for validating the yield model to ensure a more realistic and reliable yield model in extrapolation objectives.

AB - Crop yield prediction models are critical tools for evaluating growth performance and informing decisions during farm management. Developing yield prediction models that are robust not only in the ranges of the model spatial domain but also in additional locations using a data-driven approach is challenging. The main objective of this study was to investigate an appropriate cross-validation (CV) strategy for establishing transferable UAV-based yield prediction models across different spatial domains (i.e., meeting extrapolation mapping objectives). In this study, we compared three data splitting procedures for the CV protocols, including random data splitting (random CV), cluster-based spatial splitting (spatial CV), and field-specific hold-out data splitting (leave-one-field-out CV). Model optimization was also examined to determine whether these factors affect the transferability of the yield model, including performing recursive feature elimination (RFE) and comparing the effects of algorithms utilized in the yield prediction model. Three base learner algorithms, namely, random forest, XGBoost, and LASSO regression, were utilized, and a stacked ensemble technique model formed with these base learners was also implemented. The established models were later tested on an independent field as a test dataset to evaluate the model transferability performance. Random CV exhibited poor error tracking performance in predicting yield beyond the model spatial domain, while spatial CV and leave-one-field-out CV approaches provided better expectation on yield predictions outside the model's training spatial domain. Furthermore, simple models as implementing LASSO regression and RFE improved the model capability in extrapolation tasks. The results of this study suggest that spatially-aware CV should be used as the standard method rather than conventional random CV for validating the yield model to ensure a more realistic and reliable yield model in extrapolation objectives.

KW - Spatial data

KW - Leave-one-field-out

KW - Extrapolation

KW - Vegetation indices

KW - Spatial clustering

UR - http://www.scopus.com/inward/record.url?scp=85187987750&partnerID=8YFLogxK

U2 - 10.1016/j.jafr.2024.101096

DO - 10.1016/j.jafr.2024.101096

M3 - Journal article

SN - 2666-1543

JO - Journal of Agriculture and Food Research

JF - Journal of Agriculture and Food Research

M1 - 101096

ER -

Critical evaluation of the effects of a cross-validation strategy and machine learning optimization on the prediction accuracy and transferability of a soybean yield prediction model using UAV-based remote sensing

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this