Application of UAV-hyperspectral imaging for rice growth monitoring
author: cily
2023-01-18
1 Introduction
Rice is one of the three major food crops in the world. Nearly half of the world's population uses rice as a staple food. Ensuring high and stable rice yields is of great significance to world food security. Rice originated in China, and my country is still the largest rice producer and consumer in the world. The rice planting area ranks second in the world, and the total output ranks first in the world. Two-thirds of the country's population uses rice as a staple food. In the past 50 years, the annual rice planting area in the country has accounted for about 27% of the grain crop planting area, while the annual rice output has accounted for about 43% of the total grain output. It can be seen that rice occupies an extremely important position in my country's agricultural production. However, as the area of cultivated land is decreasing day by day, how to use limited land resources to obtain high-quality and high-yield rice is particularly important. Physiological parameters such as chlorophyll content, leaf area index and nitrogen are important indicators for evaluating rice growth. Rapid and accurate acquisition of physiological parameters at different growth stages of rice is of great significance for scientific fertilization and efficient field management. However, traditional determination methods are mostly based on Field sampling and laboratory chemical analysis. Although this method can accurately measure the target, it is time-consuming, laborious, and costly. It is often destructive, and it is difficult to achieve real-time monitoring in a large area. In recent years, with the development of remote sensing technology, especially the rapid development of hyperspectral remote sensing technology, it has become possible to monitor crop growth status and physiological parameters quickly, non-destructively, and in real time on a large scale, thus providing new ideas and technologies for non-destructive monitoring of crop growth support.
The growth process of rice is affected by many factors, such as cultivation mode, variety, natural factors, irrigation and fertilization treatments, etc. Chlorophyll can indicate the physiological and biochemical state of crops during growth, and it can be used to evaluate the degree of stress (such as pests and diseases, heavy metal stress on crops), growth cycle, productivity, photosynthetic capacity, etc. of crops. Not only that, the chlorophyll content is also closely related to the gross primary production (Gross Primary Production) and crop yield. Leaf area index (LAI), chlorophyll (Chl) content, and gross primary productivity (GPP) are the most common parameters in the study of vegetation biochemical parameters retrieved by remote sensing technology. The nutrient element status of crops affects the crop canopy morphology, leaf area and intrinsic physiological characteristics, so it is closely related to the spectral characteristics. In this way, massive hyperspectral data can be used to quickly, conveniently and non-destructively diagnose nutritional status in the field.
UAV Remote Sensing (UAVRS) combines unmanned aerial vehicle technology with remote sensing technology, POS positioning technology and communication technology. And analysis to provide real-time and rapid technical solutions, with strong advantages of automation, intelligence and specialization. The ATH9010NY UAV crop health index inspection system developed by our company can obtain real-time high spatial resolution and high spectral resolution remote sensing image data maps, and has the irreplaceable advantages of satellite remote sensing and ground remote sensing technology. It can provide multi-angle high-resolution images, which have obvious advantages compared with the limitations of the narrow field of view and small working range of ground remote sensing.
2. Technical ideas and main content
Use the near-ground UAV platform to obtain imaging hyperspectral data, and simultaneously carry out the collection of rice canopy spectral data and the collection of rice physiological parameters chlorophyll, leaf area index and ammonia, and then use the machine learning algorithm SVM to carry out the reflection of rice physiological parameters Finally, the inversion model was further extended and applied to UAV hyperspectral images to obtain the spatial distribution results of UAV hyperspectral rice physiological parameters. In addition, the rice growth monitoring was also carried out based on UAV hyperspectral images. Real-time monitoring of plant height and lodging, so as to better monitor rice growth and provide reference for rice planning and management. In order to realize the reuse of algorithms, all the algorithms were deployed and integrated at the end of this project, and made into a general version of the software.
What can be achieved:
1) Acquire large-scale UAV hyperspectral image data, and based on our self-developed UAV hyperspectral image data processing software, the preprocessing of hyperspectral data can be quickly realized.
2) Based on the collected rice canopy spectral data and rice physiological parameters chlorophyll, leaf area index and ammonia data, use the machine learning algorithm SVM to carry out the inversion of rice physiological parameters and obtain the inversion model.
3) Based on UAV hyperspectral images and inversion models, the spatial distribution results of UAV hyperspectral rice physiological parameters are obtained. In addition, rice plant height and lodging are extracted based on UAV hyperspectral images, so as to realize real-time rice growth. monitor.
4) Integrate and deploy all algorithms into hyperspectral image processing software to realize algorithm reuse and one-piece operation.
3 program design
3.1 Inversion of rice physiological and chemical parameters
1) SPAD: The chlorophyll of leaves was measured using SPAD-502 produced in Japan. SPAD is the abbreviation of Soiland Plant Analyzer Development, and the SPAD value is the relative content of chlorophyll, also known as greenness. The reading ranges from 0 to 99.9, and the greenness of the leaves is quantitatively described by the value. The higher the reading, the greater the chlorophyll content. Because leaf chlorophyll mainly absorbs red and blue light, it rarely absorbs infrared light. According to this principle, the SPAD instrument emits red light (peak wavelength 650 nm) and near-infrared light (peak wavelength 940 nm) to a certain part of the blade by means of the transmission method, and utilizes the difference in optical density at the two wavelengths. Measure the relative content of chlorophyll. SPAD value is dimensionless, it has a high correlation with leaf chlorophyll content, and is often used to characterize chlorophyll content.
2) Leaf area index (LAI): It is generally expressed as half of the surface area of plant leaves per unit land area. LAI is a very important structural parameter in terrestrial ecosystems. It can not only be used to estimate terrestrial ecosystems The net primary productivity of crops is also closely related to the sunlight interception, evaporation, evapotranspiration and photosynthesis of crops. At the same time, the leaf area index can be used as an important quantitative indicator of crop canopy structure changes, which will affect vegetation coverage, vegetation greenness, etc., and is often used to judge crop growth, plant community vitality, and yield estimation. The size of the crop leaf area directly affects the amount of light received inside it, and then affects the photosynthetic efficiency of the crop. On a certain land area, the larger the leaf area index, the higher the utilization rate of light energy by crops, but the larger the leaf area index, the better. When the leaf area index is too large, the leaves will cover each other, and the photosynthetic efficiency of the crop will be reversed. reduce. The leaf area index will change regularly with the replacement of plant growth seasons. Therefore, large-area
Real-time monitoring of crop LAI and its changes in space is of great significance for monitoring crop growth, crop yield estimation, and field management.
3) Ammonia: Nitrogen is the nutrient element most closely related to crop photosynthesis, yield and quality, and also the mineral element with the largest demand and application amount of crops. When crops are deficient in nitrogen, it will not only affect crop yield but also reduce its quality. On the contrary, if nitrogen nutrition is excessive, it will cause obvious non-point source pollution to water and atmosphere. This requires that in the practice of farmland management, under the premise of reducing environmental pollution as much as possible, nitrogen fertilizers should be used efficiently to obtain high-yield and high-quality agricultural products. Therefore, it is an urgent need for modern agricultural production to quickly and accurately obtain crop nitrogen status and realize precise and efficient fertilization in farmland.
4) Inversion model:
Support Vector Machine (SVM) is a machine learning method proposed by Vapnik et al. to solve classification and regression problems. It is based on statistical learning theory and structural risk minimization. It can solve the over-learning problem of small samples, and has certain advantages in the recognition of nonlinear and high-dimensional patterns. Support vector machine regression (Support Vector Regression, SVR) includes linear support vector machine regression and nonlinear support vector machine regression. The key to SVM regression lies in the determination of the kernel function. Through the kernel function, the low-dimensional nonlinear problem can be converted into a high-dimensional linear problem, and the computational complexity and results are not limited by the dimension of the input data. Therefore, SVR It can take both training accuracy and generalization ability into consideration at the same time, so that problems such as overfitting and high dimensionality can be better solved. Least squares support vector machine (LS-SVM) is an improved SVM proposed by Suykens. This method solves the complex quadratic optimization problem in SVM, and uses the partial least squares linear system as a loss function to solve a set of linear equations. . Compared with SVM, the operation speed of LS-SVM has been significantly improved. When using LS-SVM analysis, the choice of kernel function is very important. This project uses RBF kernel function:
In the formula, x is the input vector; y is the target value corresponding to x; σ2 is the RBF kernel function parameter.
Different settings of the kernel function parameters in the LS-SVM model will lead to different results, so the parameters need to be optimized. The parameters of the RBF kernel function are determined by cross-validation. The step-by-step grid search method is used in the cross-validation to reduce the search difficulty and save computing time. That is, first determine the approximate value range of the parameters, and then conduct a secondary search within this range to determine the best parameters.
5) Model checking method
In order to test the reliability of the established rice physiological parameter estimation model, according to the commonly used model evaluation methods at home and abroad, the inversion results use the following indicators to test the accuracy of the model:
3.2 Extraction of rice plant height
Plant height is defined as the distance from the base of the plant to the top of the main stem. In agriculture, plant height has a certain correlation with crop yield, lodging resistance, and photosynthetic ability. Reasonable plant height is one of the foundations for healthy growth and stable yield of crops. The plant height of crops is affected by many factors such as crop variety and nutrient status. In the identification of crop varieties and the monitoring of growth conditions, it is necessary to measure the plant height. The technical route of rice plant height extraction.
First, according to the central point coordinates of a large number of continuous single images acquired by the UAV, the images are aligned, and dense point clouds and grids are established. DSM performs fusion to obtain RGB+DSM image. Correct the plane coordinates of the RGB+DSM images according to the plane coordinates of the ground control points. After checking the elevation accuracy of the images in each period, take the elevation data of the RGB+DSM images as the benchmark, and establish an elevation fitting equation by selecting elevation correction points. Elevation correction is carried out to finally obtain DSM corrected images of each phase. Finally, the corrected DSM was used as the basis of DEM, and the plant height of rice in each breeding plot of each period was obtained through grid calculation.
3.3 Rice lodging monitoring
Lodging is a common and important factor affecting crop yields. It is often caused by meteorological disasters such as strong winds, heavy rains, and hail, and generally occurs in the later stages of crop growth. Lodging at the filling stage will affect photosynthesis of crops, resulting in insufficient accumulation of seeds and grains and dry seeds. Lodging at the mature stage will affect mechanized harvesting, and lodging crops are poorly ventilated, and the seeds are prone to mildew in a humid environment . All of these ultimately lead to severe crop loss and reduced grain quality. Rapid and accurate monitoring of crop lodging disasters will help managers assess losses in a timely manner, and also help breeders to select lodging-resistant varieties.
Random forest classification algorithm:
The Random Forests (RF) algorithm was proposed by Leo Breiman and Adele Cutler, academicians of the American Academy of Sciences in 2001. It is a machine learning classification method based on the CART decision tree, which combines multiple decision trees using the idea of ensemble learning. . Compared with a single decision tree, the random forest classification algorithm has stronger generalization ability, higher stability, and deeper training of samples, and can successfully avoid the overfitting phenomenon caused by a single decision tree, even if It can also maintain good stability in the case of small sample data volume or unbalanced sample size distribution.
In this rice lodging monitoring, the spectral characteristics, texture characteristics, spectral index and color characteristics of rice are used as the classification raw data of normal rice and lodging rice, which are imported into the random forest algorithm for accurate extraction of lodging rice, so as to realize the accurate and accurate detection of lodging rice. Quick extraction, in addition, the crop lodging monitoring model is essentially a classification model, and the confusion matrix is used to evaluate the classification accuracy of the model. In the confusion matrix, the four indicators of Kappa, total accuracy, user accuracy and mapping accuracy are used to measure the classification effect. The closer these four indicators are to 1, the higher the consistency between the classification result and the ground truth.
Rice is one of the three major food crops in the world. Nearly half of the world's population uses rice as a staple food. Ensuring high and stable rice yields is of great significance to world food security. Rice originated in China, and my country is still the largest rice producer and consumer in the world. The rice planting area ranks second in the world, and the total output ranks first in the world. Two-thirds of the country's population uses rice as a staple food. In the past 50 years, the annual rice planting area in the country has accounted for about 27% of the grain crop planting area, while the annual rice output has accounted for about 43% of the total grain output. It can be seen that rice occupies an extremely important position in my country's agricultural production. However, as the area of cultivated land is decreasing day by day, how to use limited land resources to obtain high-quality and high-yield rice is particularly important. Physiological parameters such as chlorophyll content, leaf area index and nitrogen are important indicators for evaluating rice growth. Rapid and accurate acquisition of physiological parameters at different growth stages of rice is of great significance for scientific fertilization and efficient field management. However, traditional determination methods are mostly based on Field sampling and laboratory chemical analysis. Although this method can accurately measure the target, it is time-consuming, laborious, and costly. It is often destructive, and it is difficult to achieve real-time monitoring in a large area. In recent years, with the development of remote sensing technology, especially the rapid development of hyperspectral remote sensing technology, it has become possible to monitor crop growth status and physiological parameters quickly, non-destructively, and in real time on a large scale, thus providing new ideas and technologies for non-destructive monitoring of crop growth support.
The growth process of rice is affected by many factors, such as cultivation mode, variety, natural factors, irrigation and fertilization treatments, etc. Chlorophyll can indicate the physiological and biochemical state of crops during growth, and it can be used to evaluate the degree of stress (such as pests and diseases, heavy metal stress on crops), growth cycle, productivity, photosynthetic capacity, etc. of crops. Not only that, the chlorophyll content is also closely related to the gross primary production (Gross Primary Production) and crop yield. Leaf area index (LAI), chlorophyll (Chl) content, and gross primary productivity (GPP) are the most common parameters in the study of vegetation biochemical parameters retrieved by remote sensing technology. The nutrient element status of crops affects the crop canopy morphology, leaf area and intrinsic physiological characteristics, so it is closely related to the spectral characteristics. In this way, massive hyperspectral data can be used to quickly, conveniently and non-destructively diagnose nutritional status in the field.
UAV Remote Sensing (UAVRS) combines unmanned aerial vehicle technology with remote sensing technology, POS positioning technology and communication technology. And analysis to provide real-time and rapid technical solutions, with strong advantages of automation, intelligence and specialization. The ATH9010NY UAV crop health index inspection system developed by our company can obtain real-time high spatial resolution and high spectral resolution remote sensing image data maps, and has the irreplaceable advantages of satellite remote sensing and ground remote sensing technology. It can provide multi-angle high-resolution images, which have obvious advantages compared with the limitations of the narrow field of view and small working range of ground remote sensing.
2. Technical ideas and main content
Use the near-ground UAV platform to obtain imaging hyperspectral data, and simultaneously carry out the collection of rice canopy spectral data and the collection of rice physiological parameters chlorophyll, leaf area index and ammonia, and then use the machine learning algorithm SVM to carry out the reflection of rice physiological parameters Finally, the inversion model was further extended and applied to UAV hyperspectral images to obtain the spatial distribution results of UAV hyperspectral rice physiological parameters. In addition, the rice growth monitoring was also carried out based on UAV hyperspectral images. Real-time monitoring of plant height and lodging, so as to better monitor rice growth and provide reference for rice planning and management. In order to realize the reuse of algorithms, all the algorithms were deployed and integrated at the end of this project, and made into a general version of the software.
What can be achieved:
1) Acquire large-scale UAV hyperspectral image data, and based on our self-developed UAV hyperspectral image data processing software, the preprocessing of hyperspectral data can be quickly realized.
2) Based on the collected rice canopy spectral data and rice physiological parameters chlorophyll, leaf area index and ammonia data, use the machine learning algorithm SVM to carry out the inversion of rice physiological parameters and obtain the inversion model.
3) Based on UAV hyperspectral images and inversion models, the spatial distribution results of UAV hyperspectral rice physiological parameters are obtained. In addition, rice plant height and lodging are extracted based on UAV hyperspectral images, so as to realize real-time rice growth. monitor.
4) Integrate and deploy all algorithms into hyperspectral image processing software to realize algorithm reuse and one-piece operation.
3 program design
3.1 Inversion of rice physiological and chemical parameters
1) SPAD: The chlorophyll of leaves was measured using SPAD-502 produced in Japan. SPAD is the abbreviation of Soiland Plant Analyzer Development, and the SPAD value is the relative content of chlorophyll, also known as greenness. The reading ranges from 0 to 99.9, and the greenness of the leaves is quantitatively described by the value. The higher the reading, the greater the chlorophyll content. Because leaf chlorophyll mainly absorbs red and blue light, it rarely absorbs infrared light. According to this principle, the SPAD instrument emits red light (peak wavelength 650 nm) and near-infrared light (peak wavelength 940 nm) to a certain part of the blade by means of the transmission method, and utilizes the difference in optical density at the two wavelengths. Measure the relative content of chlorophyll. SPAD value is dimensionless, it has a high correlation with leaf chlorophyll content, and is often used to characterize chlorophyll content.
2) Leaf area index (LAI): It is generally expressed as half of the surface area of plant leaves per unit land area. LAI is a very important structural parameter in terrestrial ecosystems. It can not only be used to estimate terrestrial ecosystems The net primary productivity of crops is also closely related to the sunlight interception, evaporation, evapotranspiration and photosynthesis of crops. At the same time, the leaf area index can be used as an important quantitative indicator of crop canopy structure changes, which will affect vegetation coverage, vegetation greenness, etc., and is often used to judge crop growth, plant community vitality, and yield estimation. The size of the crop leaf area directly affects the amount of light received inside it, and then affects the photosynthetic efficiency of the crop. On a certain land area, the larger the leaf area index, the higher the utilization rate of light energy by crops, but the larger the leaf area index, the better. When the leaf area index is too large, the leaves will cover each other, and the photosynthetic efficiency of the crop will be reversed. reduce. The leaf area index will change regularly with the replacement of plant growth seasons. Therefore, large-area
Real-time monitoring of crop LAI and its changes in space is of great significance for monitoring crop growth, crop yield estimation, and field management.
3) Ammonia: Nitrogen is the nutrient element most closely related to crop photosynthesis, yield and quality, and also the mineral element with the largest demand and application amount of crops. When crops are deficient in nitrogen, it will not only affect crop yield but also reduce its quality. On the contrary, if nitrogen nutrition is excessive, it will cause obvious non-point source pollution to water and atmosphere. This requires that in the practice of farmland management, under the premise of reducing environmental pollution as much as possible, nitrogen fertilizers should be used efficiently to obtain high-yield and high-quality agricultural products. Therefore, it is an urgent need for modern agricultural production to quickly and accurately obtain crop nitrogen status and realize precise and efficient fertilization in farmland.
4) Inversion model:
Support Vector Machine (SVM) is a machine learning method proposed by Vapnik et al. to solve classification and regression problems. It is based on statistical learning theory and structural risk minimization. It can solve the over-learning problem of small samples, and has certain advantages in the recognition of nonlinear and high-dimensional patterns. Support vector machine regression (Support Vector Regression, SVR) includes linear support vector machine regression and nonlinear support vector machine regression. The key to SVM regression lies in the determination of the kernel function. Through the kernel function, the low-dimensional nonlinear problem can be converted into a high-dimensional linear problem, and the computational complexity and results are not limited by the dimension of the input data. Therefore, SVR It can take both training accuracy and generalization ability into consideration at the same time, so that problems such as overfitting and high dimensionality can be better solved. Least squares support vector machine (LS-SVM) is an improved SVM proposed by Suykens. This method solves the complex quadratic optimization problem in SVM, and uses the partial least squares linear system as a loss function to solve a set of linear equations. . Compared with SVM, the operation speed of LS-SVM has been significantly improved. When using LS-SVM analysis, the choice of kernel function is very important. This project uses RBF kernel function:
In the formula, x is the input vector; y is the target value corresponding to x; σ2 is the RBF kernel function parameter.
Different settings of the kernel function parameters in the LS-SVM model will lead to different results, so the parameters need to be optimized. The parameters of the RBF kernel function are determined by cross-validation. The step-by-step grid search method is used in the cross-validation to reduce the search difficulty and save computing time. That is, first determine the approximate value range of the parameters, and then conduct a secondary search within this range to determine the best parameters.
5) Model checking method
In order to test the reliability of the established rice physiological parameter estimation model, according to the commonly used model evaluation methods at home and abroad, the inversion results use the following indicators to test the accuracy of the model:
3.2 Extraction of rice plant height
Plant height is defined as the distance from the base of the plant to the top of the main stem. In agriculture, plant height has a certain correlation with crop yield, lodging resistance, and photosynthetic ability. Reasonable plant height is one of the foundations for healthy growth and stable yield of crops. The plant height of crops is affected by many factors such as crop variety and nutrient status. In the identification of crop varieties and the monitoring of growth conditions, it is necessary to measure the plant height. The technical route of rice plant height extraction.
First, according to the central point coordinates of a large number of continuous single images acquired by the UAV, the images are aligned, and dense point clouds and grids are established. DSM performs fusion to obtain RGB+DSM image. Correct the plane coordinates of the RGB+DSM images according to the plane coordinates of the ground control points. After checking the elevation accuracy of the images in each period, take the elevation data of the RGB+DSM images as the benchmark, and establish an elevation fitting equation by selecting elevation correction points. Elevation correction is carried out to finally obtain DSM corrected images of each phase. Finally, the corrected DSM was used as the basis of DEM, and the plant height of rice in each breeding plot of each period was obtained through grid calculation.
3.3 Rice lodging monitoring
Lodging is a common and important factor affecting crop yields. It is often caused by meteorological disasters such as strong winds, heavy rains, and hail, and generally occurs in the later stages of crop growth. Lodging at the filling stage will affect photosynthesis of crops, resulting in insufficient accumulation of seeds and grains and dry seeds. Lodging at the mature stage will affect mechanized harvesting, and lodging crops are poorly ventilated, and the seeds are prone to mildew in a humid environment . All of these ultimately lead to severe crop loss and reduced grain quality. Rapid and accurate monitoring of crop lodging disasters will help managers assess losses in a timely manner, and also help breeders to select lodging-resistant varieties.
Random forest classification algorithm:
The Random Forests (RF) algorithm was proposed by Leo Breiman and Adele Cutler, academicians of the American Academy of Sciences in 2001. It is a machine learning classification method based on the CART decision tree, which combines multiple decision trees using the idea of ensemble learning. . Compared with a single decision tree, the random forest classification algorithm has stronger generalization ability, higher stability, and deeper training of samples, and can successfully avoid the overfitting phenomenon caused by a single decision tree, even if It can also maintain good stability in the case of small sample data volume or unbalanced sample size distribution.
In this rice lodging monitoring, the spectral characteristics, texture characteristics, spectral index and color characteristics of rice are used as the classification raw data of normal rice and lodging rice, which are imported into the random forest algorithm for accurate extraction of lodging rice, so as to realize the accurate and accurate detection of lodging rice. Quick extraction, in addition, the crop lodging monitoring model is essentially a classification model, and the confusion matrix is used to evaluate the classification accuracy of the model. In the confusion matrix, the four indicators of Kappa, total accuracy, user accuracy and mapping accuracy are used to measure the classification effect. The closer these four indicators are to 1, the higher the consistency between the classification result and the ground truth.
Precision fertilization by UAV for rice at tillering stage in cold region based on hyperspectral remote sensing prescription map 
Application of hyperspectral camera in industrial inspection 
Estimation Scheme of Rice Yield Based on UAV Hyperspectral Images 
Surveillance of pine wood nematode disease based on satellite remote sensing images 
UAV forest fire patrol protection plan 
Application of Hyperspectral Imager in Disguised Target Recognition(part 2) 
Application of hyperspectral imager in detection of exogenous pests in jujube fruit 
Application of UAV Hyperspectral in Garbage Sorting
