Two storage devices environment

Description of the environement

This example simulates the operation of a realistic micro-grid (such as a smart home for instance) that is not connected to the main utility grid (off-grid) and that is provided with PV panels, batteries and hydrogen storage. The battery has the advantage that it is not limited in instaneous power that it can provide or store. The hydrogen storage has the advantage that is can store very large quantity of energy.

python run_MG_two_storage_devices

This example uses the environment defined in MG_two_storage_devices_env.py. The agent can either choose to store in the long term storage or take energy out of it while the short term storage handle at best the lack or surplus of energy by discharging itself or charging itself respectively. Whenever the short term storage is empty and cannot handle the net demand a penalty (negative reward) is obtained equal to the value of loss load set to 2euro/kWh.

The state of the agent is made up of an history of two to four punctual observations:

  • Charging state of the short term storage (0 is empty, 1 is full)
  • Production and consumption (0 is no production or consumption, 1 is maximal production or consumption)
  • (Distance to equinox)
  • (Predictions of future production : average of the production for the next 24 hours and 48 hours)

Two actions are possible for the agent:

  • Action 0 corresponds to discharging the long-term storage
  • Action 1 corresponds to charging the long-term storage
More information can be found in
Deep Reinforcement Learning Solutions for Energy Microgrids Management, Vincent François-Lavet, David Taralla, Damien Ernst, Raphael Fonteneau

Annex to the paper

PV production and consumption profiles

Solar irradiance varies throughout the year depending on the seasons, and it also varies throughout the day depending on the weather and the position of the sun in the sky relative to the PV panels. The main distinction between these profiles is the difference between summer and winter PV production. In particular, production varies with a factor 1:5 between winter and summer as can be seen from the measurements of PV panels production for a residential customer located in Belgium in the figures below.

http://vincent.francois-l.be/img_GeneralDeepQRL/ProductionVSMonths_be.png

Total energy produced per month

http://vincent.francois-l.be/img_GeneralDeepQRL/ProductionVSTime_1janv_be.png

Typical production in winter

http://vincent.francois-l.be/img_GeneralDeepQRL/ProductionVSTime_1july_be.png

Typical production in summer

A simple residential consumption profile is considered with a daily average consumption of 18kWh (see figure below).

http://vincent.francois-l.be/img_GeneralDeepQRL/ConsumptionVSTime_random.png

Representative residential consumption profile

Main microgrid parameters

Data used for the PV panels
cost \(c^{PV}\) \(1 euro/W_p\)
Efficiency \(\eta^{PV}\) \(18 \%\)
Life time \(L^{PV}\) \(20 years\)
Data used for the \(LiFePO_4\) battery
cost \(c^B\) \(500 euro/kWh\)
discharge efficiency \(\eta_0^B\) \(90\%\)
charge efficiency \(\zeta_0^B\) \(90\%\)
Maximum instantaneous power \(P^B\) \(> 10kW\)
Life time \(L^{B}\) \(20 years\)
Data used for the Hydrogen storage device
cost \(c^{H_2}\) \(14 euro/W_p\)
discharge efficiency \(\eta_0^{H_2}\) \(65\%\)
charge efficiency \(\zeta_0^{H_2}\) \(65\%\)
Life time \(L^{H_2}\) \(20 years\)
Data used for reward function
cost endured per kWh not supplied within the microgrid \(k\) \(2 euro/kWh\)
revenue/cost per kWh of hydrogen produced/used \(k^{H_2}\) \(0.1 euro/kWh\)