approaches aimed to extract specific handcrafted features
(e.g., spectral reflectivity and vegetation indices (
VIs
)) from
hyperspectral images to recognize plants under the drought
condition. The main problem of these methods is the feature
selection. Spectral reflectivity,
VIs
and other indices, which
have been widely used as features in the previous studies [21]
[16], are pixelwise calculations of pixel intensities in the indi-
vidual hyperspectral image. In other words, they are all pixel-
wise features that represent crop growing conditions at one
time instant. However, as the plant is growing, the drought
stress condition should be a continuous procedure with a
unique time-series variation pattern, which can be well repre-
sented by temporal features. Thus, compared to the individual
image, a time-series image sequence containing the temporal
information will be a better representation of the plant grow-
ing condition for the early drought stress detection task.
Several previous studies have shown a relationship between
the leaf water stress and the spectral reflectance variation in
the visible region [13][4]. This investigation indicates that the
RGB
image is able to be used for the early drought stress detec-
tion task. In other words, considering the temporal features, the
early drought stress detection problem can be formulated as
the classification of time-series
RGB
image sequences contain-
ing drought plants or not. Compared to previous approaches
using hyperspectral images, the methods based on
RGB
image
analysis are more cost-effective for the consumer applications.
In recent years, long short-term memory (
LSTM
) networks
have been widely used in different real world classification
tasks, such as action recognition [17][15], event detection [18]
[8] and natural language processing [33][32]. As an effective
method to uncover the hidden temporal relation in time-lapse
data and classify the sequential data,
LSTM
is suitable for our
task of early drought stress detection.
In this study we proposed a Bidirectional Long Short-Term
Memory (
BLSTM
) model to solve the problem of early drought
plant stress detection on
RGB
images. First, we extract the
time-series image patch sequences that contain the temporal
variation information of the plant as patch sequences. Second-
ly, a pre-trained Convolutional Neural Network model is used
for extracting discriminative features from each image in the
patch sequences. Finally, the patch sequence, in the form of a
sequence of feature vectors, will be input to the
BLSTM
for the
binary classification. Two independently collected datasets
are used to validate the performance of our proposed method.
The main contributions of this work are: (i) the application
of
BLSTM
to
RGB
image sequences for early drought plant stress
detection for the first time, (ii) the investigation of the earliest
moment that we can detect the plant drought stress condition
from
RGB
images, and (iii) the proposal of an efficient
RGB
image
data collection strategy that can use less time and manpower
for the purpose of accurate early drought plant stress detection.
The rest of this paper is organized as follows: in the next
section, we introduce the methodology of our proposed meth-
od including data acquisition, data preparation, the proposed
BLSTM
model, and the proposal of an efficient
RGB
image
data collection strategy; Next, we discuss the data collection
guidance and validate our method on two
RGB
image datasets
followed by our paper’s conclusions.
Methodology
In this section, we first introduce the two plant image data-
sets tested in this work; then, illustrate the pipeline of our
proposed method, including the data preparation step and the
BLSTM
model. Finally, we introduce the
RGB
image collection
strategy for the drought plant stress detection.
Data Acquisition
Two independently collected
RGB
image datasets of crop
plants are utilized in this work. The
LemnaTecDD Dataset
is
collected from the LemnaTec platform [31] at the Donald
Danforth Plant Science Center. In the LemnaTec platform,
plants are automatically transported by conveyers through a
series of imaging cabinets to capture images from two sides,
as well as, from the above. To collect this dataset, 10 repli-
cates of 27 nested association mapping* (
NAM
) lines of maize
*
Nested Association Mapping (
nam
) is a technique designed for identify-
ing and dissecting the genetic architecture of complex traits in corn [36].
Figure 1. Image samples in the
LemnaTecDD Dataset
.
460
July 2018
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
