Quoted from: Couillet, Romain, Denis Trystram, and Thierry Ménissier. “The submerged part of the AI-ceberg.” IEEE Signal Processing Magazine, September 2022.
The energy consumption of a single training run of the latest (by 2020) deep neural networks dedicated to natural language processing exceeds 1,000 megawatt-hours (more than a month of computation on today’s most powerful clusters). This corresponds to an electricity bill of more than 100,000 euros (figures in the millions of euros are sometimes found) and 500 tons of CO2 emissions – that is, the carbon footprint equivalent to 500 transatlantic round trips from Paris to New York. In comparison, the human brain consumes in a month about 12 kWh, i.e., a hundred thousand times less, for tasks much more complex than natural language translation.
Unlike a mere ten years ago and in spite of the improvement in desktop computer capabilities, it is no longer possible today to train a modern neural network on a personal computer (it would theoretically take up to 405 years)… One may object that it is probably not surprising that deep learning algorithms be far less energy efficient than three billion years of biological evolution and that the figures may rather suggest a huge room for potential improvement… This objection would displace the focus of the point made here: in a matter of ten years, the absolute consumption of AI learning skyrocketed to reach levels of hundreds of tons of equivalent CO2 for a single learning task. These levels are at stunning odds with the requirements for the human society to drastically reduce its carbon footprint at a rate of −7%/year, starting today.
Let us pursue on the objection line of argument: “once trained”, the algorithm can be reused billions of times, improving billions of users’ satisfaction and well-being, at a comparatively negligible cost on individual devices; this would make the initial investment, however large, possibly worth it, if not desirable. The argument here makes the untold assumption that AI algorithms do improve human welfare, which is already a debated position, but also hides (i) the fact that practical R&D in AI is a continuous process of trials and errors of deep architectures run on an increasing amount of dedicated servers and (ii) that new algorithms run on up-to-date devices or even dedicated devices, thereby making former equipment obsolete and enforcing the continuous purchase of new terminals.
Quoted from: Couillet, Romain, Denis Trystram, and Thierry Ménissier. “The submerged part of the AI-ceberg.” IEEE Signal Processing Magazine, September 2022.
