Every Chat with AI Has a Carbon Cost — Here’s What You Should Know

OpenAI CEO Sam Altman recently said that an average ChatGPT query consumes about as much energy as an oven running for just over a second. That might sound small, until you multiply it by millions of prompts every single day.

But what does “average” even mean?

Sasha Luccioni, climate lead at AI platform Hugging Face, put it simply: “You can’t just throw a number out there.” It all depends on how much computing goes on behind the scenes.

Think of an AI model like a super-brain with billions or even trillions of “knobs,” known as parameters. The more knobs it has, the better it can learn, but also, the more energy it takes to run.

Models like GPT-4 are too big for your phone or laptop. They sit inside huge data centers packed with high-powered chips called GPUs. These GPUs crunch data at lightning speed to answer your questions in real time, and they gulp electricity to do it.

Already, 4.4% of all U.S. energy is used by data centers. And by 2028, that number could nearly triple.

Most companies won’t tell us how much energy they use to train their AI models. The process can take weeks or months and burn through thousands of chips. And this results in a mystery cloud of carbon emissions.

Even the part we do see, when you use the chatbot (called “inference”), isn’t easy to measure. It depends on where the data center is located, what type of electricity it runs on, and even what time of day you’re chatting.

“Energy has been the middle child that nobody wants to talk about,” says Mosharaf Chowdhury, a computer scientist at the University of Michigan.

To dig deeper, researchers in Germany tested 14 open-source AI models using NVIDIA’s A100 chip. The models that gave longer, step-by-step explanations used more than 500 extra words (or “tokens”) per question than standard ones, meaning more energy, more emissions.

And let’s not forget the embodied carbon: all the emissions it takes to build the machines, chips, and buildings in the first place.

Don’t worry, you don’t have to give up your favorite chatbot forever. But there are small changes that can make a big difference.

Use smaller models for simple tasks. Not every question needs a “super-brain.” Maximilian Dauner, one of the researchers involved in the study, pointed out that smaller AI models are perfectly capable of handling simple questions. By choosing these more efficient models for basic tasks, it’s possible to significantly cut down on CO₂ emissions without compromising on performance.

Avoid extra words. That polite “please” or “thank you” might warm the AI’s digital heart, but it also uses more computing time. 

Avoid peak energy hours. Using AI during cooler times (like late nights) or outside peak electricity usage windows helps reduce the load on power grids, much like setting your AC wisely.

If you’re a developer or researcher, consider using tools like Hugging Face’s AI Energy Score or ML. Energy, which help track and compare how much energy different models use. 

Right now, AI is growing faster than our ability to measure or manage its energy use. But the solution may lie in smarter policies. Luccioni suggests a system like appliance energy ratings: if your AI tool serves over 10 million people daily, it must meet a minimum efficiency score.

Because while AI might be invisible to the eye, its impact on our planet is very, very real.

Luccioni cautions that if tech companies fail to adjust their energy practices soon, the consequences could quickly spiral out of control.

So the next time your AI assistant crafts a poem or plans your grocery list, remember, every word comes with a cost. Let’s make sure it’s one the planet can afford.

For more details, refer to this article published in Frontiers.