h
t
=
f
(
W
1
x
t
+
W
r
x
t
–1
+
b
1
)
(1)
y
t
=
g
(
W
2
h
t
+
b
2
)
(2)
where
W
j
and
b
j
represent the input weight matrix and bias
vector of the
j
th
hidden layer, respectively, and
W
r
is denoted
as a recurrent weight matrix;
f
and
g
represent activation
functions of the hidden layer and output layer, respectively.
The traditional
RNN
, especially trained with gradient de-
scent, has a significant problem called the
vanishing gradient
[12]. This problem causes traditional
RNN
to forget informa-
tion after just a few steps. Long short-term memory (
LSTM
), a
special kind of architecture
RNN
that remembers information
for long periods of time, is designed to overcome the vanish-
ing gradient problem [25]. In
LSTM
, the hidden cells of the tra-
ditional
RNN
are replaced by memory blocks. Therefore,
LSTM
is capable to find and exploit long range dependencies in the
sequential data. Normally, each
LSTM
memory block consists
of a memory cell and three gates: the input gate, the output
gate and the forget gate. These gates control the information
flow in the
LSTM
block. As shown in Figure 6, the forget gate
can reset the cell variable by forgetting the stored input
c
t
,
while the input and output gates are in charge of reading
input from
x
t
and writing output to
h
t
, respectively:
c
t
=
f
t
7
c
t
–1
+
i
t
7
tanh(
W
xc
x
t
+
W
hc
h
t
–1
+
b
c
)
(3)
h
t
=
o
t
7
tanh(
c
t
)
(4)
where
7
denotes element-wise multiplication and tanh is the
hyperbolic tangent function
that is also applied in an ele-
ment-wise manner.
i
t
,
o
t
and
f
t
are representing the output of
the input gate, output gate and forget gate, respectively, while
b
c
is a bias term and
W
is the weight matrix. Since each
LSTM
memory block is an independent unit, the activation vectors
i
t
,
o
t
,
f
t
and
c
t
are all of same size as
h
t
. Note, each gate is only
dependent on the cell within the same memory block.
To solve the early drought plant stress detection task, in
addition to using
LSTM
memory blocks to remember infor-
mation for a long period of time, we adopt the bidirectional
mechanism from the bidirectional
RNN
(
BRNN
) [24] to design
a bidirectional
LSTM
(
BLSTM
) model for processing the input
sequential data in both temporal directions. The motivation of
bringing the bidirectional mechanism to our scheme is to en-
able our proposed model to have a better understanding of the
unique variation pattern of drought plants. Because the water
content variation of the plant in the early stage is subtle to be
recognized, especially for those plants with the mild drought
stress, such as the plants with 50%
FC
or 75%
FC
watering
regimes in the
LemnaTecDD Dataset
. Compared with the
LSTM
model that can only process input data in one direction,
the proposed
BLSTM
model, which can process input data in
forward direction and backward direction respectively, is
capable of exploring the full context information of the water
content variation for identifying subtle differences between
drought plants and normal plants.
The proposed
BLSTM
model is depicted in Figure 7. In the
input-layer, each hidden layer has K
LSTM
blocks, each of
which takes a feature vector in the feature vector sequence as
the input. In
LSTM
blocks, gates are activated using the
stan-
dard sigmoid function,
(1 +
e
–
x
)
–1
, and the block input as well
as the block output is squashed with the
hyperbolic tangent
function
(tanh). After presenting an input sequence entirely
to the
BLSTM
, the result can be read at the output-layer. The
time-lapse feature vector sequence, which is the represen-
tation of a patch sequence, will be classified as either the
drought condition or the control condition.
To train the
BLSTM
model, feature vector sequences ex-
tracted from patch sequences containing drought plants are
considered as positive samples (the drought condition), while
feature vector sequences extracted from patch sequences con-
taining control plants are negative samples (the control condi-
tion). Both of these two kinds of samples will be fed to the
model. The proposed
BLSTM
model is built using Python on
the Keras [3] toolkit with a Tensorflow backend. The weights
of the network are initialized from a
N
(0.0, .05) distribution,
while we add 1 to the forget gate bias of LSTMs at initializa-
tion. The network is trained with an
RMSprop
optimizer,
where the learning rate is 10
–3
with the decay rate as 10
–6
. The
applied dropout probability in our network is 0.5. We train
the model for 50 epochs and use early stopping based on the
validation performance.
Optimal Data Collection Strategy
In addition to the
BLSTM
model for the drought plant stress
detection, we first propose the Optimal Data Collection Strat-
egy (
ODCS
) that can use less information to accurately detect
the plant drought stress from
RGB
images as early as possible.
The image data collection for the drought stress detection
task is expensive, especially in crop fields. In order to acquire
detailed variation information of the growing plants, we have
to collect the image data from the first few days after planting
as frequently as possible, which is inefficient. To eliminate
Figure 6. The structure of the
LSTM
block.
Figure 7. The architecture of the proposed
BLSTM
model that
takes the time-lapse feature vector sequence as the input
for the final classification. Each time-lapse feature vector
sequence is representing a patch sequence.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
July 2018
463
