PE&RS July 2018 Full - page 461

are planted in a randomized complete block design with four
different watering regimes (25% FC †, 50% FC, 75% FC and
100% FC). Starting at the 15th day after planting (DAP), plants
are imaged daily for ten days (15th to 24 th DAP ) (image samples
are shown in Figure 1). The image resolution is 2,454 × 2,056
pixels. The experiment is repeated four times. Therefore,
there are 1,080 (10 × 27 × 4) plants involved in this dataset.
Each plant has three image sequences (two side view se-
quences and one top view sequence), each of which is
considered as an independent plant sample. In this work,
those image sequences containing plants with 25% FC , 75%
FC or 50% FC watering regimes will be considered as the
drought samples, and image sequences containing plants with
100% FC watering regimes will be the control samples.
The MSTCivil Dataset is collected in the greenhouse of the
Civil, Architectural, and Environmental Engineering Depart-
ment in Missouri University of Science and Technology. In
this dataset, there are three kinds of crops: drought tolerant
maize, drought-susceptible maize, and sorghum. For each
kind of crop, 16 replicates are planted, where 8 replicates
are grown under the drought stress condition and the other 8
replicates are under the control condition. In the greenhouse,
there are four RGB cameras installed on the ceiling to collect
image data from the top (image samples are shown in Figure
2). Camera 1 and Camera 2 are for the drought groups. Camera
3 and Camera 4 are for the control groups. The image resolu-
tion is 1280×1040 pixels. Starting from the first day of plant-
ing, the four cameras image the plants hourly from 6 a.m. to
5 p.m. (12 hours) every day for 30 days. For all the images
collected by a certain camera, those images captured at the
same time instant will be grouped into an image sequence.
For example, for all the images collected by Camera 1 in 30
days, those images taken at 6 a.m. will be grouped into an im-
age sequence that contains 30 time-lapse images. Therefore,
in the MSTCivil Dataset, there are 48 (four cameras, hourly
over 12 hours) image sequences, each of which contains 30
images over 30 days.
Data Preparation
The data preparation process, as illustrated in Figure 3, in-
cludes two steps (extracting patch sequences from the image
sequence and extracting feature descriptions from the im-
ages), which aims to transform the image data of plants into
a form that is suitable for our proposed BLSTM model for the
final classification for drought and non-drought plants.
Patch Sequence Extraction
Given a plant sample in the form of a time-lapse image sequence
that contains K time-lapse images, we first downsize the images
to 324 × 324 pixels. Then, a 3D sliding window is applied on
the downsized image sequence (dimension: 324 × 324 × 3K‡) to
crop image patch sequences that contain a part of the plant. The
size of the 3D sliding window is set as 224 × 224 × 3K with the
stride size of 10 pixels. After the patch sequence extraction step,
we obtain time-lapse patch sequences, each of which can be
considered as a patch sequence for the classification.
In this study, instead of focusing on the entire plant, we are
more interested in the temporal variation pattern in the patch
sequence that only contains a part of the plant. Whereas the
entire plant is able to provide the morphological information
that helps detecting the drought plant, in practical cases, it is
†fc, field capacity, is the amount of soil moisture or water content
held in the soil after excess water has drained away and the rate of
downward movement has decreased [14].
‡In the downsized image sequence, there are k rgb images, each of
which can be represented by a 324 × 324 × 3 matrix. Thus, the down-
sized image sequence can be represented as a 324 × 324 × 3K matrix,
where each rgb image has three channels.
Figure 2. Image samples in the MSTCivil Dataset.
Figure 3. The illustration of the data preparation process on the LemnaTecDD Dataset. The same data preparation process is
also applied on the MSTCivil Dataset.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING July 2018 461