Solution recommendation | Retrieval of vegetation nitrogen and water content based on UAV hyperspectral

Preface

With the continuous advancement of my country's ecological civilization construction, the development of urban ecological civilization has received more and more attention. Urban vegetation is a part of building green development, and its deployment is an important indicator of urban ecological civilization. Monitoring of vegetation growth is the basis for the sustainable and healthy development of urban greening. Water, as an indispensable condition for the normal growth of vegetation, plays an important role in the normal growth of vegetation. Nitrogen is an important element in vegetation growth activities. It is essential for the synthesis of vegetation chlorophyll, proteins, enzymes and other substances, and plays a key role in vegetation light synthesis. The growth of plants is inseparable from water, light, nutrients and other conditions. Changes or lack of one of these conditions will cause the appearance of the vegetation to show different characteristics. The appearance of the vegetation is closely related to its growth pattern. Therefore, accurate monitoring of vegetation water content and nitrogen has a great impact on the normal growth of vegetation. Accurately monitoring the nitrogen and water content of urban vegetation will help scientific management of urban vegetation, promote the healthy growth of urban vegetation, and accelerate the construction of urban ecological civilization.

Technical ideas and main content

Optosky uses a nationally produced hyperspectral imager to monitor vegetation nitrogen elements and water content. It mainly obtains imaging hyperspectral data through a near-ground drone platform and combines it with a fieldspec spectroradiometer to establish a vegetation leaf database, and then carry out water content and nitrogen detection based on deep convolutional neural network, extracting information, conducting sensitive band analysis, and finally achieving accurate detection based on PNN probabilistic neural network algorithm, realizing identification and detection of vegetation nitrogen and water content, and providing more efficient and intelligent methods for monitoring of healthy growth of vegetation.  thereby promoting the construction of urban ecological civilization. Taking vegetation moisture content as an example, the figure below shows the flow chart for extracting leaf moisture content.

Figure 1 Inversion process of vegetation leaf water content.

What needs to be implemented:

1) Develop vegetation observation plans and collection standards, establish urban vegetation data sets in the natural environment through fieldspec spectroradiometer collection and preprocessing, and preprocess the collected original vegetation images to improve image contrast and enhance the difference between vegetation and non-vegetation parts of the image, improve the accuracy of vegetation information.

2) In order to solve the problem of cluttered background and strong interference in vegetation aerial images collected by drones, the vegetation information extraction method was studied. The vegetation information was extracted by fusing the texture features and color features of the vegetation on the drone aerial images and combining it with threshold segmentation information to separate it from non-vegetation information.

3) In order to solve the problems of traditional hyperspectral inversion of vegetation water content and nitrogen high cost and poor applicability, a vegetation inversion model based on visible light is proposed. HSCNN technology and color matrix are used to extract image spectral features, and then regression method is used to establish spectral features and the mapping relationship between actual vegetation water content and nitrogen enables automatic identification of urban vegetation water content and nitrogen, and evaluates the accuracy of the model.

4) The drone flies once a month, based on long-term sequence monitoring image data, thereby providing a reference for accurate monitoring.

Technical points

●Convolutional neural network

If a traditional neural network (Figure 2) uses a fully connected network structure, that is, a convolutional neural network is connected to each neuron in the network and adjacent layers, a large number of parameters will be required. It is often used to solve nonlinear problems.

Figure 2 Neural network structure diagram.

Convolutional neural network is a feedforward neural network that contains convolutional calculations and has a deep structure. It is one of the most commonly used representative algorithms of deep learning. It uses deep convolution to simulate the hierarchical perception and local receptive fields in the human visual perception mechanism to process some unstructured data and integrate low, medium and high different levels of features in an end-to-end manner, thereby obtaining rich feature information and improving the accuracy of semantic segmentation of remote sensing images. The basic structure of a convolutional neural network generally includes a data input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer. Its basic model framework is shown in Figure 3.

Figure 3 Basic model framework of convolutional neural network.

●HSCNN

HSCNN is a unified deep learning framework. Specifically, RGB images are first upsampled in spectral dimensions by simple interpolation, and then the proposed method learns end-to-end mapping from a large number of upsampled/terrestrial hyperspectral image pairs. The mapping is represented as a deep convolutional neural network (CNN), which takes the spectrally upsampled image as input and predicts the lost image details from which the final hyperspectral image is recovered. The HSCNN network structure is shown in Figure 4.

Figure 4 HSCNN network structure diagram (the network consists of three processes: patch extraction, feature mapping and reconstruction).

●Calculation formulas for various vegetation indices

Program implementation

Hyperspectral drone flight service

Hyperspectral reflects the characteristics of high-resolution optical information, which uses many very narrow electromagnetic wave bands (usually <10nm) to obtain relevant data from objects of interest. Hyperspectral images are acquired by an imaging spectrometer, which provides tens to hundreds of narrow-band spectral information for each pixel, producing a complete and continuous spectral curve. Image information can reflect the external quality characteristics of the sample such as size, shape, defects, etc. Since different components have different spectral absorption, the image will have a more significant reflection on a certain defect at a specific wavelength, and the spectral information can fully reflect differences in internal physical structure and chemical composition of the sample. Therefore, hyperspectral characteristics can be used to separate vegetation from non-vegetation, and combined with drones. Efficient water content and nitrogen element retrieval can be achieved.

Figure 5 UAV hyperspectral image consisting of 300 bands.

UAV hyperspectral has the following characteristics:

① There are many spectral features. The imaging spectrometer has 300 bands in the visible and near-infrared spectral regions;

②High spectral resolution. The sampling interval of the imaging spectrometer is small, and the resolution is less than 3nm. Fine spectral resolution reflects the subtle characteristics of the spectrum of ground objects;

③ The amount of data is rich. As the number of bands increases, the amount of data increases exponentially. It can provide spatial domain information and spectral domain information, that is, "unification of images and spectra", and the spectral curve obtained by the imaging spectrometer can be compared with the spectral curve of similar ground objects measured on the ground.

In the planned research area, hyperspectral images are collected according to needs. Specific steps are as follows:

(1) UAV side settings: Assemble the UAV hyperspectral equipment, set the altitude and speed, and set the image route spacing according to the camera parameters and image overlap requirements;

(2) Camera side settings: Set the camera frame rate according to the altitude and speed, and set the integration time (exposure time) according to the whiteboard measurement value;

(3) Standard reflectivity whiteboard: Place a standard reflectivity whiteboard in the route area, and the whiteboard needs to be captured when collecting images.

Figure 6 Schematic diagram of UAV hyperspectral data collection.

UAV hyperspectral image processing

After collecting UAV hyperspectral image data, the following preprocessing work is required:

(1) Wavelength calibration: The original image collected does not have wavelength information, and a wavelength calibration file needs to be added;

(2) Image cropping: Hyperspectral uses push-broom imaging, which requires cropping of the collected images of the measurement area;

(3) Registration and splicing: perform geo-registration or relative registration on the cropped survey area images, and then splice the registered images into a complete image;

(4) Radiation correction: The value in the original image represents the reflection intensity, which needs to be corrected using the whiteboard reflection value and standard reflectance to calculate the reflectance of the entire image.

(5) Mixed spectral decomposition: The spectral data collected by drones are affected by the spatial resolution of the image, resulting in a pixel that may be mixed and averaged by different ground objects or vegetation. In order to improve the accuracy, a mixed spectral decomposition operation needs to be performed on the image;

(6) Spectral filtering (smoothing): There is a certain amount of noise in the spectral information in the original image, and spectral filtering needs to be performed before application.

Spectral image processing software interface:

Figure 7 Schematic diagram of UAV hyperspectral image processing software interface.

Ground object sample data collection

Optosky uses nationally produced hyperspectral imagers and fieldspec spectroradiometers to conduct on-site surveys. The purpose is to provide real training samples for on-site classification models and test samples for classification results for the classification of UAV remote sensing images. The main content of the on-site investigation is to measure the spectral data of vegetation leaves, and classify all the spectral data collected by the ground object spectrometer into different levels as a standard training sample data set for UAV hyperspectral image data processing.

Figure 8 Schematic diagram of fieldspec spectroradiometer equipment and data collection.

Result analysis

Figure 9 Nitrogen inversion result chart.

Figure 10 Vegetation moisture content inversion results.

1. Reliability of estimation results

Model accuracy testing can be analyzed based on two indicators: Root Mean Square Error (RMSE) and correlation coefficient (r).

In the formula, n is the number of samples, S_i^- is the laboratory measurement value of sample i, Y_i^- and is the average value of all verification samples; S_i is the predicted value using the model, and Y_i is the average value of the predicted values of all corresponding verification samples. The smaller the RMSE value and the larger the R correlation coefficient value, the higher accuracy of the model.

2. Factors affecting estimation accuracy

The factors that affect the accuracy of the inversion results mainly include the following aspects:

① Spectral data: Due to environmental influences and uncertain human operations during field data collection, there are certain differences in the quality of hyperspectral images captured. But this factor is not the main factor.

② Actual measurement data: Actual measurement data includes spectral data and parameter quantitative measurement data of leaves. Affected by instrumentation and human operating factors, these two measured data may have certain errors, but they will have little impact on the inversion results. Importantly, the time difference between the two measured values is one of the main factors in the image inversion model. Try to avoid measuring leaves for a long time after picking them, and ensure that spectral measurement and parameter quantitative measurement are carried out at the same time.

③ Inversion model: The construction of the inversion model is the main factor affecting the accuracy of the inversion. Affected by the selection of characteristic spectra, the form of variables, and the form of the inversion model, there is great uncertainty in the inversion results.