Leveraging Large Language Models and GAN-Augmented Data for Energy Load Prediction

Abstract

Large Language Models (LLMs) originally developed for tasks like text generation and translation have shown successful potential in capturing temporal and complex dependencies, making them suitable for different tasks. In this study, we want to propose a LLMs-based Heating Load (HL) and Cooling Load (CL) estimation model based on residential building characteristics. At first, a prompt generation module was proposed to convert in-hand tabular data to useful prompts, and then the hugging face pre-trained Bart-base model was re-trained to create a new prediction tool for residential buildings HL and CL prediction. In addition, to improve the performance of the proposed LLM-based model, a new data augmentation module was proposed based on Generative Adversarial Network (GAN) and Conditional GAN to increase the size of training data. The proposed model combines Data Augmentation and Prompt generation Modules with LLM and is named DAPM-LLM. The prediction result showed that the DAPM-LLM can predict energy usage using linguistic prompts, and the data augmentation module improved model performance by 600% and 300% in HL and CL prediction, respectively. The comparison of its results with other works shows its superiority over most of them except ensemble models. Using larger pre-trained models and sufficient data will enable these models to outperform ensemble models too. The results showed that the DAPM-LLM model can be successfully used in solving complex problems such as energy consumption prediction, and can be used by engineers and designers to select the best design/plan for building construction by using linguistic sentences.