As shown in Table 3 and Table 4, the proposed
BLSTM
mod-
el outperforms all the other models in both of the patch se-
quence and image sequence classification evaluations on the
two plant image datasets. Since the
CNN
model only considers
the spatial features in the individual image patch without any
temporal variation information, it does not work well with the
drought samples in the early stage. Compared with the
CNN
model, the
LSTM
model is able to use the variation informa-
tion during the plant growth to identify the drought samples
in the early stage. However, due to the subtle variation of
the plants with the mild drought condition (some samples
shown in Figure 8), the
LSTM
model will make mistakes in the
classification of the mild drought samples. Compared with
the
LSTM
model, by using the bidirectional mechanism, the
proposed
BLSTM
model and the
BRNN
model, which can learn
the full context information in the temporal variation pattern,
are able to recognize most of the mild drought samples. The
BLSTM
model is slightly better than the
BRNN
model in the
classification performance. But the
BRNN
model needs more
training time to obtain a good classification performance than
the
BLSTM
model, due to the vanishing gradient problem.
Compared to the competitive classification performances
on the
LemnaTecDD Dataset
, the
BLSTM
model achieves
inferior performance (but still competitive compared to other
methods) in the
MSTCivil Dataset
. The main reason is the
strong interference in the
MSTCivil Dataset
(shown in Figure
9), which is collected in a greenhouse that needs to be used
for several experiments simultaneously. In some images, some
plants are out of view for recording measurements, which
cause inconsistency to the image data. Since the image data
are collected using the natural light source, shadows and non-
uniform illumination conditions add challenging interferenc-
es to the early drought plant stress detection task. In addition
to these two main problems, some other interferences, such as
occlusions and water stain reflections, add nonuniform noises
to the image data, which are also challenging problems to the
drought plant stress detection task on
RGB
images.
Conclusions
Early drought plant stress detection is of great relevance
in precision plant breeding and production. However, the
previous methods based on hyperspectral image analysis
were mostly focusing on analyzing the relationship between
spectral reflectivity and the leaf water content on individual
hyperspectral images, but they ignored the temporal variation
information of the plants under the drought stress condition.
In addition, the applications of these approaches are limited
by the high cost of hyperspectral imaging systems. In this
work, we apply the Bidirectional Long Short-Term Memory
(
BLSTM
) networks to
RGB
image datasets for early drought
plant stress detection for the first time. By using
LSTM
memory
blocks and the bidirectional mechanism, the proposed
BLSTM
model is able to use the discriminative temporal variation
information and the full context information in the early
plant growth stage for the classification of a patch sequence
containing plants under the drought condition or not. Two
independently collected
RGB
image datasets are used for the
validation of the proposed method. Optimal data collection
strategies in a given environment are also investigated to ef-
ficiently detect the drought stress in the early stage.
Acknowledgments
This work is supported by NSF CAREER award IIS-1351049,
NSF EPSCoR Grant IIA-1355406, Intelligent System Center
and Center of Biomedical Science and Engineering at Mis-
souri University of Science and Technology.
References
1. Boyer, J., 1982. Plant productivity and environment,
Science
,
218(4571):443–448.
2. Bengio, Y., et al., 2012. Deep learning of representations for
unsupervised and transfer learning,
Proceedings of ICML
Workshop on Unsupervised and Transfer Learning
, pp. 17–36.
Figure 9. Image sequence samples in the
MSTCivil Dataset
. Samples with illumination and human interferences are
challenging problems to our proposed method.
466
July 2018
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
