Competition Outline

Energy consumption forecasting is crucial in current and future power and energy systems. With the increasing penetration of renewable energy sources, with high associated uncertainty due to the dependence on natural conditions, such as wind speed or solar intensity, the need to balance the fluctuation of generation with the flexibility from the consumer side increases considerably. In this way, significant work has been done on the development of energy consumption forecasting methods, able to deal with different forecasting circumstances, e.g., the prediction time horizon, the available data, the frequency of data, or even the quality of data measurements. The main conclusion is that different methods are more suitable for different prediction circumstances, and there it is not clear that a certain method can outperform all others in all situations.
This competition aims at bringing together the most recent advances in building energy consumption forecasting methods. Authors of methods that present the best results in this competition will be invited as panelists in the Panel:

Performance evaluation of artificial intelligence methods for energy consumption forecasting using open data sets

Jointly organized by the IEEE PES AMPS/ISS

To take place at the 2021 IEEE Power & Energy Society General Meeting to be held 25-29 July 2021 in Washington, DC

https://pes-gm.org/2021/

Competition Format and Important Dates

The competition will be held during a full business week (14-18 June, 2021). Each day of this week participants will be asked to provide their consumption forecasts for the following day. Immediately after the deadline for submission, the real data from the respective day will be provided, so that it can be used to generate the next output.
In order to build and refine the forecasting models, a Full Year of historical data (1st data set release) will be provided by mid-April (participants need to register and login to have access to these data).
2 weeks before the competition week, by 31st May , the next 40 days of historical (2nd data set release) data will be provided – referring to the period imediately before the days to be forecasted. Finally, during the competition week, the data that refers to the last day is provided on a daily basis (3rd data set release).

All the details on the data sets will be provided when the data is available. In summary:
Requested forecasts: Total building consumption of every 15 minutes during the following day.
Inputs (all referring to 5 minutes intervals): Total building consumption; consumption per building area; external temperature, weather information, PV generation.

The provided values of power, included in data sets, represent the mean of instant power during 5 minutes. To participate in this competition, it is requested the forecasted values representing the mean of instant power during 15 minutes, leaving to the participant the required processing of data needed to provide results for 15 minutes periods.

The forecasted values that participants must provide are the consumption of the building (without the forecast or the consideration of local generation).

Submisson details

During the competition week, the forecasts for the upcoming day must be submitted in XLS format. The file should be named: PARTICIPANTLASTNAME_DATE.xls and the file should contain 2 columns: time and consumption forecast; and 96 rows: one for each period of 15 minutes during the 24 hours of the day.

The submission of the daily forecasts files should be done in two ways: 1 – upload in the competition website (after login) and 2 – send by email to the organizers (tcp@isep.ipp.pt, zav@isep.ipp.pt, log@isep.ipp.pt, hmori@meiji.ac.jp), with the XLS file attached (NO links, e.g. dropbox, onedrive, allowed).

The email subject should be: ODS-Competition_2021_LastName_DDMMYYYY

Submissions must be done until 16:00 (i.e. 4 p.m.) UTC+1 (London time). Only submissions received (via website and email) until this time will be considered.
The daily data set referring to the current day (which can be used to generate the next output) will be provided one hour later: by 17:00 (i.e. 5 p.m.) UCT+1.

Evaluation metric

A specific evaluation metric has been defined for this competition, which rewards low forecasting errors during the periods of highest activity (higher consumption) and small variations of forecasting errors.

The competition Evaluation Metric is described as follows:

– The final score is calculated through the average of the 5 scores referring to each competition day.

For each day:

The absolute errors of all 96 daily forecasts are calculated.
The Standard Deviation (SD) of the 96 errors is calculated.
All errors from hours 07 to 20 (highest activity hours) are penalized (absolute error is doubled).
The 12 highest errors during hours 07 to 20 (highest activity hours) are identified and penalized (absolute error x 4).
The Mean Absolute Error (MAE) comprising the 96 daily forecasts (with penalties) + the 12 penalized highest errors is calculated.
The daily score is calculated as the weighted average between the calculated MAE and the SD. The weight for the MAE is 5 and for the SD is 1.

Task Force on Open Data Sets (ODS)

https://site.ieee.org/psace-ods/

Working group on Intelligent Data Mining and Analysis (IDMA)

https://site.ieee.org/psace-idma/

Intelligent Systems Subcommittee (ISS)

IEEE Power and Energy Systems Society

Data set description

The building used in this challenge is divided in five zones, where each zone has 2 to 3 offices. The building has photovoltaic panels with a generation peak of 7.5 kW, a near weather station where the data was collected, inside sensors, light intensity control, and all consumption measured by load type. The data was collected in 5 minutes periods.
By zones, the buildings have a distribution of 6 researchers in Zone#1, 5 researchers in Zone#2, 5 researchers in Zone#3, 3 researchers in Zone#4, and 5 researchers in Zone#5. Zone#1 includes a meeting room, and Zone#2 includes a server room. Regarding the server room, its HVAC unit was measured in HVAC#2, however, near the end of the year the unit was removed from the monitoring system.
The light intensity values represent the current state of the lamps and have a linear correlation with the lamp’s consumption.
The data set represents raw data without any treatment, this means that it is possible to find errors. The weather data can have missing data or missing reading periods. The remaining data do not have missing reading periods, but they can have a fixed zero (0) value, indicating a failure in the system. You can assume that periods with zero voltage represent an error in that zone at that period.

To be released during the competition

Disclaimer

The data sets provided in the scope of the ‘competition on building energy consumption forecasting’ are for competition
purposes only. All information on the data sets are provided in good faith and cannot be used outside the competition
scope. However, we can provide part of these data, or alternative/additional data, by request, to be used in research
works. To request data for purposes outside the current competition, please contact the competition organizers listed in
http://www.gecad.isep.ipp.pt/smartgridcompetitions/, describing the objectives of your work and the data needed.

Submission

The forecasts for the upcoming day must be submitted in XLS format. The file should be named: PARTICIPANTLASTNAME_DATE.xls and the file should contain 2 columns: time and consumption forecast; and 96 rows: one for each period of 15 minutes during the 24 hours of the day.

The submission of the daily forecasts files should be done in two ways: 1 – upload in this page (after login) and 2 – send by email to the organizers (tcp@isep.ipp.pt, zav@isep.ipp.pt, log@isep.ipp.pt, hmori@meiji.ac.jp), with the XLS file attached (NO links, e.g. dropbox, onedrive, allowed).