1st African Conference on Precision Agriculture | 8-10 December | 2020

#7660 A REINFORCEMENT LEARNING BASED APPROACH FOR EFFICIENT

IRRIGATION WATER MANAGEMENT

El Hachimi Chouaib1, Belaqziz Salwa1,2, Khabba Said1,3 and Chehbouni Abdelghani1,4

1
Mohammed VI Polytechnic University (UM6P), Center for Remote Sensing Applications
(CRSA), Benguerir, Morocco; chouaib.elhachimi@um6p.ma; 2 Faculty of Sciences d’Agadir,
Ibn Zohr University, Agadir, Morocco; salwa.belaqziz@um6p.ma; 3 LMFE, Faculty of
Sciences Semlalia, Cadi Ayyad University, Marrakech, Morocco; khabba@uca.ac.ma; 4
Centre d’Etudes Spatiales de la BIOsphère (CESBIO), Toulouse, France;
ghani.chehbouni@ird.fr

ABSTRACT

Due to population growth and the effects of climate change, most of the world's regions
are threatened by water scarcity, especially in Africa and Mediterranean region. In Morocco,
the agriculture consumes more than 85% of available water. Thus, to preserve water resources,
the rational management of irrigation water is necessary. In this context, recent technological
progress and the emergence of artificial intelligence could provide an effective decision support
tool for the rational and sustainable use of this resource. In this paper, we propose an approach
based on reinforcement learning, a type of machine learning that uses trial-and-error principle
to learn how to best fit situation to action in a highly dynamic, stochastic environment. In this
proposed approach, a Farmer Agent learns to choose the optimal cropping pattern defined by
the type of crop, area to cultivate, sowing date and irrigation plan depending on the water
availability at the beginning of the agricultural season. Each agent interacts with the
environment which is composed of environmental and socio-economic modules containing
different processes to provide the Farmer Agent with the information he needs to learn. The
use of reinforcement learning in this complex system will certainly change the traditional
irrigation water management mode and bring more intelligence into the system. This approach
will be generalized in future work to cover the entire agricultural sector and study the behavior
of many Farmer Agents. This will be used then, for the design and implementation of a decision
support system platform that can be used at the beginning of the agricultural season to make
informed decisions.

Keywords: water resources management, irrigation water management, reinforcement

learning, precision agriculture, machine learning, cropping pattern optimization.

INTRODUCTION

Water resources are already under pressure [1]. Most regions of the world are
experiencing water stress. This is due to several factors such as population growth, rising living
standards, and climate change which intensifies dangerous phenomena such as heavy rains in
some areas and drought in others due to the disruption of the normal water cycle. The latter
represents the supply of water to the environment. Water resources problems arise at demand
levels. In Morocco, 15% of water resources are used by industry for the cooling of machines
and for production, or tourism which uses it for swimming pools, and irrigation of golf fields
and for domestic use. The remaining 85% is used by the agricultural sector mainly for irrigation
activities [2]. In addition, over 76% of irrigation in Morocco is gravity-fed and uses non-
optimal methods and techniques.

328
 1st African Conference on Precision Agriculture | 8-10 December | 2020

Having a better combination of crop choice, optimal area to cultivate, sowing dates,
and profitability is a real problem for farmers. It is currently done in a non-optimal way. Thus,
we propose in this paper an approach that uses reinforcement learning to address this
optimization problem.
LITERATURE REVIEW

Several studies have tried to solve the problem of efficient irrigation water management
with different approaches. Roque et al. [3] used supervised learning with the objective of
reproducing the behavior of an expert agronomist by regressing the amount of water to be
pumped into the local basin at the following week. They used different types of data collected
from sensors installed in orchards and meteorological data. Meanwhile, Evangelos et al. [4]
have proposed a method for optimal management of irrigation water in the city of Athena. This
method is divided into two phases. The phase of training during which a neural network learns
based on historical weather data and the constraints associated with each reservoir of the city.
And an application phase during which the system will be able to perform optimal operations
for each new state. With such approaches, the performance of the model will be linked directly
to the data annotation process which depends on annotator reasoning. It can therefore never
exceed the level of expertise of the agronomist who annotated the data for the first case and the
environmental characteristics for the second case. In recent years, reinforcement learning has
shown promising results as an optimization method. Most of their applications are only in
games such as Elhadji et al. [5] where Deep Reinforcement Learning was applied in the Pong
game, testing whether two agents can dynamically learn to divide their area of responsibility
and avoid collisions in front of a hard-coded adversary. Elhadji et al used several architectures,
namely DQN, DQN with double Q-learning, and Dueling network architectures.
We propose in this paper to apply the Reinforcement Learning to a real-world problem,
where an agent learns to maximize his objective function independently of a supervisor.

PROPOSED METHOD

Our approach is based on reinforcement learning where a Farmer Agent learns from its
environment through actions and feedbacks. Each time he observes the situation, selects an
action and gets feedback (reward/punishment) (Figure 1).
The environment is characterized by a state belonging to S, the set of states. This
environment changes depending on the interaction with the Farmer Agent. It provides him with
observations (Ot) at each instant t. These observations represent the current state of the
environment.
Then, the Farmer Agent decides based on these observations. And the environment will
be affected by that decision and provides him with a reward (Rt) in case of a positive results
(increasing the profit), or a punishment otherwise (exceed the amount of allocated water,
deficit). This repetitive process will enable the Farmer Agent to evaluate his actions according
to his final objective.

329
 1st African Conference on Precision Agriculture | 8-10 December | 2020

Figure 1. The proposed model architecture

The environment is composed of two modules: the environmental module which is

responsible for managing meteorological data, water availability and soil information, and the
socio-economic module that provides the agent with all information needed such as crops water
needs, growth phases durations, seed prices, and fertilizers prices. It also allows the agent to
calculate his incomes at the end of the agricultural season. After the training phase, the Farmer
Agent will be able to choose an optimal cropping pattern defined by the type of crop, the area
to be cultivated, the sowing date, and the irrigation plan depending on the amount of available
water and initial constraints (Figure 2).

Figure 2. Inputs and outputs of the final system

330
 1st African Conference on Precision Agriculture | 8-10 December | 2020

The mathematical modeling of our problem uses the Markov Decision Process [6],
which represents the basis of reinforcement learning. The goal of solving MDP is to find an
optimal policy that will maximize the sum of the expected rewards (Equation 1). The Bellman
equation (Equation 1) is called the value function. It is decomposed into two parts: an
immediate reward R(s, a), and a discounted value of the successor state.

𝑉 (𝑠) = 𝑚𝑎𝑥s t𝑅(𝑠, 𝑎) + 𝛾 A 𝑃(𝑠, 𝑎, 𝑠′)𝑉(𝑠′){

yz
Equation 1. Bellman equation
Where:
S: the set of states which is water availability and environment’s constraints (Figure 2),
A: the set of actions (a combination of a crop type, area to cultivate, sowing date and irrigation
plan),
P: the set of transition’s probabilities,
R: A reward in case of a positive income, and a punishment in case of exceeding water
availability or initial constraints.
λ: discount factor.

CONCLUSIONS

In this paper, we have proposed an approach based on reinforcement learning for the
agricultural sector at the crop field level. It will provide farmers with the best cropping pattern,
defined by the type of crop, area to cultivate, sowing date, and irrigation plan depending on the
water availability at the beginning of the agricultural season and initial constraints. This work
will form the basis for developing a decision support platform to help farmers make informed
decisions about their agricultural practices. The next phase will be at the level of the entire
agricultural sector where we will generalize and study the interactions between many agents
with different objectives.

REFERENCES

[1] Richey AS, Thomas BF, Lo M-H, Reager JT, Famiglietti JS, Voss K, Swenson S, Rodell
M. 2015. “Quantifying renewable groundwater stress with GRACE”, Water Resour.
Res. 51:5217- 5238, doi:10.1002/2015WR017349.
[2] Hssaisoune M, Bouchaou L, Sifeddine A, Bouimetarhan I, Chehbouni A. 2020. “Moroccan
groundwater resources and evolution with global climate changes,” Geosci. doi:
10.3390/geosciences10020081.
[3] Torres-Sanchez R, et al. 2020. “A decision support system for irrigation management:
Analysis and implementation of different learning techniques” Water (Switzerland),
doi: 10.3390/w12020548.
[4] Rozos E. 2019. “Machine Learning, Urban Water Resources Management and Operating
Policy,” Resources, doi: 10.3390/resources8040173.
[5] Diallo EAO, Sugiyama A, Sugawara T. 2020. “Coordinated behavior of cooperative agents
using deep reinforcement learning,” Neurocomputing, doi:
10.1016/j.neucom.2018.08.094.
[6] Bellman R. 1957. “A Markovian Decision Process,” Indiana Univ. Math. J. 6(4):679-684.

331