# Two storage devices environment¶

## Description of the environement¶

This example simulates the operation of a realistic micro-grid (such as a smart home for instance) that is not connected to the main utility grid (off-grid) and that is provided with PV panels, batteries and hydrogen storage. The battery has the advantage that it is not limited in instaneous power that it can provide or store. The hydrogen storage has the advantage that is can store very large quantity of energy.

python run_MG_two_storage_devices


This example uses the environment defined in MG_two_storage_devices_env.py. The agent can either choose to store in the long term storage or take energy out of it while the short term storage handle at best the lack or surplus of energy by discharging itself or charging itself respectively. Whenever the short term storage is empty and cannot handle the net demand a penalty (negative reward) is obtained equal to the value of loss load set to 2euro/kWh.

The state of the agent is made up of an history of two to four punctual observations:

• Charging state of the short term storage (0 is empty, 1 is full)
• Production and consumption (0 is no production or consumption, 1 is maximal production or consumption)
• (Distance to equinox)
• (Predictions of future production : average of the production for the next 24 hours and 48 hours)

Two actions are possible for the agent:

• Action 0 corresponds to discharging the long-term storage
• Action 1 corresponds to charging the long-term storage
Deep Reinforcement Learning Solutions for Energy Microgrids Management, Vincent François-Lavet, David Taralla, Damien Ernst, Raphael Fonteneau

## Annex to the paper¶

### PV production and consumption profiles¶

Solar irradiance varies throughout the year depending on the seasons, and it also varies throughout the day depending on the weather and the position of the sun in the sky relative to the PV panels. The main distinction between these profiles is the difference between summer and winter PV production. In particular, production varies with a factor 1:5 between winter and summer as can be seen from the measurements of PV panels production for a residential customer located in Belgium in the figures below.

Total energy produced per month

Typical production in winter

Typical production in summer

A simple residential consumption profile is considered with a daily average consumption of 18kWh (see figure below).

Representative residential consumption profile

### Main microgrid parameters¶

 cost $$c^{PV}$$ $$1 euro/W_p$$ Efficiency $$\eta^{PV}$$ $$18 \%$$ Life time $$L^{PV}$$ $$20 years$$
 cost $$c^B$$ $$500 euro/kWh$$ discharge efficiency $$\eta_0^B$$ $$90\%$$ charge efficiency $$\zeta_0^B$$ $$90\%$$ Maximum instantaneous power $$P^B$$ $$> 10kW$$ Life time $$L^{B}$$ $$20 years$$
 cost $$c^{H_2}$$ $$14 euro/W_p$$ discharge efficiency $$\eta_0^{H_2}$$ $$65\%$$ charge efficiency $$\zeta_0^{H_2}$$ $$65\%$$ Life time $$L^{H_2}$$ $$20 years$$
 cost endured per kWh not supplied within the microgrid $$k$$ $$2 euro/kWh$$ revenue/cost per kWh of hydrogen produced/used $$k^{H_2}$$ $$0.1 euro/kWh$$