this issue, we propose the
ODCS
that aims
to use as less image data as possible in
a time period that is informative for the
early drought plant stress detection task.
To find the
ODCS
, we design differ-
ent data sampling strategies on the plant
image dataset to simulate different data
collection strategies. However, it is unprac-
tical to try all possible sampling strategies,
which will be a huge number of experi-
ments testing the proposed method using
different combinations of the images in
the time-lapse patch sequence. Therefore,
we select 15 representative data sampling
strategies which can cover most of the time
periods in the time-lapse patch sequence.
We intuitively design the sequence
lengths of the data sampling strategies as
10 images, 15 images, and 20 images. The
data sampling strategy is named as S
m
F
p
L
q
,
which means the original time-lapse patch
sequence is sampled with time interval m
from the p
th
(First) image to the q
th
(Last)
image in the sequence. For instance,
the S
1
F
11
L
30
strategy, which samples the
original time-lapse patch sequence from
the 11
th
image to the 30
th
image with time
interval 1, is simulating the data collection strategy that col-
lects plant image data every other day from the 11
th
day to
the 30
th
day after planting. If the S
1
F
11
L
30
strategy can achieve
competitive classification result on the early drought plant
stress detection task, then, instead of collecting data every
day for 30 days, we can use a lower data collection frequency
by imaging the plant every other day during 20 days to save
the manpower and the time. Note, in a data sampling strategy
S
m
F
p
L
q
, if the time interval m = 0, we will select the consecu-
tive image patches from the p
th
image to the q
th
image in the
original time-lapse patch sequence. These 15 sampling strate-
gies are summarized in Table 1.
The proposed
BLSTM
model will be tested by using the
sampled data sequences generated by different strategies. Cor-
respondingly, in order to take input data in different lengths,
the layout of the proposed
BLSTM
architecture will be adjusted
by varying the number of
LSTM
blocks. The classification per-
formances of these strategies will be compared and discussed
to find the
ODCS
.
Experiments
The main goals of this work are: (i) the application of the
proposed
BLSTM
model to
RGB
images for the early drought
plant stress detection task, (ii) the investigation of the earli-
est moment that we can accurately detect the drought plant
stress condition from
RGB
images, and (iii) the proposal of an
efficient
RGB
image data collection strategy that can reduce the
amount of time and manpower and guarantee the accuracy of
early drought plant stress detection at the same time.
To validate the first goal, the proposed
BLSTM
model is
compared with the bidirectional
RNN
(
BRNN
) model, the
LSTM
model, and the
CNN
model. For the second and third goals, we
design different data sampling strategies and compare their
classification performances to find the Optimal Data Collec-
tion Strategy (
ODCS
).
In this section, we first describe evaluation metrics. Then,
we compare different sampling strategies to find the
ODCS
.
Finally, we validate the effectiveness of our proposed method
on the
LemnaTecDD Dataset
and the
MSTCivil Dataset
, re-
spectively.
Evaluation Metrics
We adopt the leave-one-out policy in the experiment. In each
dataset, the image data will be evenly separated into four sub-
sets, where three subsets are used for training and the last one
is for testing. We perform the leave-one-out experiment four
times with each subset as the testing set alternatively. Then, the
average performance on the four experiments in terms of preci-
sion, recall and F-score is utilized as the evaluation metrics.
In the experiments, we evaluate both the
patch sequence
classification
and the
image sequence classification
. The
patch sequence classification
is the
BLSTM
model’s prediction
on the input patch sequence. In the data preparation step, by
applying the 3D sliding window (dimension: 224 × 224 × 3K)
on the image sequence (dimension: 324 × 324 × 3K) with the
stride size of 10 pixels, the image sequence is decomposed
into 100 patch sequences. Therefore, the
image sequence clas-
sification
is voted by its 100 decomposed patch sequences.
For instance, if more than half of these patch sequences are
classified as the drought condition, then the image sequence
will be classified as the drought condition.
One of the comparison methods, the
CNN
method, takes
the individual patch as the input and performs patch-wise
classification rather than sequence-wise classification. There-
fore, to evaluate the
patch sequence classification
of the
CNN
method, the patch sequence is voted by its image patches in
the sequence. For instance, given a patch sequence, if more
than half of its image patches are classified as the drought
condition by the
CNN
method, then it will be classified as the
drought condition. Similarly, we define the
image sequence
classification
for the
CNN
method.
Experiments on Optimal Data Collection Strategies (ODCS)
Compared to the
LemnaTecDD Dataset
that only collects
plant image data daily during ten days, the
MSTCivil Data-
set
, which collects image data daily during 30 days, is more
suitable for the experiments of finding the
ODCS
. Therefore, by
using the 15 selected data sampling strategies, the proposed
BLSTM
model is tested on the
MSTCivil Dataset
. The classifica-
tion performance of these strategies are presented in Table 2.
According to the classification results shown in Table 2,
there are three main observations:
Table 1. The description of the 15 data sampling strategies. The data sampling
strategy is named as S
m
F
p
L
q
, which means that the data sequence is sampled with
time interval m from the p
th
(First) image to the q
th
(Last) image in the sequence.
Sequence Length Name
Description
10 images
S
0
F
1
L
10
Sampled with time interval 0 from 1
st
image to 10
th
image.
S
0
F
11
L
20
Sampled with time interval 0 from 11
th
image to 20
th
image.
S
0
F
21
L
30
Sampled with time interval 0 from 21
st
image to 30
th
image.
S
1
F
1
L
20
Sampled with time interval 1 from 1
st
image to 20
th
image.
S
1
F
6
L
25
Sampled with time interval 1 from 6
th
image to 25
th
image.
S
1
F
11
L
30
Sampled with time interval 1 from 11
th
image to 30
th
image.
S
2
F
1
L
30
Sampled with time interval 2 from 1
st
image to 30
th
image.
15 images
S
0
F
1
L
15
Sampled with time interval 0 from 1
st
image to 15
th
image.
S
0
F
11
L
25
Sampled with time interval 0 from 11
th
image to 25
th
image.
S
0
F
16
L
30
Sampled with time interval 0 from 16
th
image to 30
th
image.
S
1
F
1
L
30
Sampled with time interval 1 from 1
st
image to 30
th
image.
20 images
S
0
F
1
L
20
Sampled with time interval 0 from 1
st
image to 20
th
image.
S
0
F
6
L
25
Sampled with time interval 0 from 6
th
image to 25
th
image.
S
0
F
11
L
30
Sampled with time interval 0 from 11
th
image to 30
th
image.
30 images
S
0
F
1
L
30
Sampled with time interval 0 from 1
st
image to 30
th
image.
464
July 2018
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
