Why do training and operating large artificial intelligence models use so much energy?

The high energy consumption associated with training and operating large artificial intelligence (AI) models is primarily driven by the intensive computational demands of the hardware required to process these workloads.12

AI data centers utilize dense clusters of graphics processing units (GPUs) rather than standard central processing units, as GPUs are specifically designed to handle the massive, complex calculations necessary for training and running large language models and machine-learning models. These components operate continuously and generate significant amounts of heat, which necessitates robust, energy-intensive cooling infrastructure to prevent equipment damage and ensure optimal performance.12

Beyond the direct power required for computing and cooling, the scale of these facilities is immense; some modern AI data centers possess power capacities reaching hundreds of megawatts. As the reliance on generative AI grows, the global power demand for these facilities has surged, leading to concerns regarding their environmental impact and strain on electrical grids. Consequently, some operators are exploring renewable energy sources or on-site power generation to mitigate their reliance on traditional electrical grids.12