Power Hungry Processing: Watts Driving the Cost of AI Deployment?
Source: ArXiv.org
In your opinion, between a generalist AI model and a specialised AI model, which of the 2 AI models consumes the most energy? This was the question posed by researchers Alexandra Sasha Luccioni, Yacine Jernite and Emma Strubell.
Figure 1 - The 5 modalities examined in our study, with the number of parameters of each model on the x axis and the average amount of carbon emitted for 1000 inferences on the y axis.
NB: Both axes are in logarithmic scale.
To obtain the answer, the researchers analysed the consumption of generative AI models that performdifferent tasks:
Text classification;
Token classification;
Extractive question answering;
Masked language modeling;
Text generation;
Summarization;
Image classification;
Object detection;
Image captioning and image generation.
For the software, the researchers used the CodeCarbon application (open-source software for measuring the energy impact of Python code). For the hardware, the researchers used a service from Amazon Web Service (AWS) based on an NVIDIA A100-SXM4-80GB GPU.