Publication
Controlled Multi-modal Image Generation for Plant Growth Modeling
Miro Miranda Lorenz; Lukas Drees; Ribana Roscher
In: 26TH International Conference on Pattern Recognition. International Conference on Pattern Recognition (ICPR-2022), located at 26TH International Conference on Pattern Recognition, August 21-25, Montreal, Quebec, Canada, IEEE, 2022.
Abstract
Predicting plant development is an important task in precision farming and an essential metric for decision-making by researchers and farmers. In this work, we propose a novel generative modeling technique for plant growth prediction based on conditional generative dversarial networks. We formulate plant growth as an image-to-image translation task and predict the appearance of a plant growth stage as a function of its previous stage. We take into account that plant growth is inherently multi-modal, depending on numerous and highly variable environmental factors, and thus a single input belongs to a distribution of potential outputs. We encode the ambiguity in an interpretable and low-dimensional latent vector space representing the various factors of variation that are infuencing plant growth. We use a novel encoder-based data fusion technique and combine information contained in remote sensing imagery of different cropping systems with data containing the factors of variation to adequately model plant growth. This offers several advantages over existing methods: (1) we show that we can model a distribution of potential appearances and simultaneously outperform existing methods in providing more realistic predictions, (2) the complexity of plant growth is more adequately captured, as various factors infuencing plant growth can be
included, (3) predictions are controllable by being conditioned by an interpretable latent vector representing the factors of variation along with an input image of a previous growth stage