As machine learning experiments become more sophisticated, their carbon footprint grows. Now researchers have calculated the carbon cost of training a number of models in cloud computing data centers in different locations. Their findings could help researchers reduce emissions created by work that relies on artificial intelligence (AI). The team found significant differences in emissions between geographic locations. In the same AI experiment, “the most efficient regions produced about a third of the emissions of the least efficient,” says Jesse Dodge, a machine learning researcher at the Allen Institute for Artificial Intelligence in Seattle, Washington, who led the study.
Until now, there have been no good tools for measuring emissions produced by cloud-based AI, says PriyaDonti, a machine learning researcher at Carnegie Mellon University in Pittsburgh, Pennsylvania, and co-founder of the Climate Change AI group.“This is great work by great authors and contributes to an important dialogue about how machine learning workloads can be managed to reduce their emissions,” he says.
Dodge and his collaborators, which included researchers from Microsoft, tracked electricity consumption while training 11 common artificial intelligence models, from the types of language models that underlie Google Translate to vision algorithms that automatically label images. They combined this data with estimates of how emissions from the electrical grids that power 16 Microsoft Azure cloud servers change over time to calculate the energy use of training in a number of locations.
Facilities in different locations have different carbon footprints due to global differences in energy sources as well as fluctuations in demand. The team found that training BERT, a common language machine learning model, in data centers in the central United States or Germany emitted 22–28 kilograms of carbon dioxide, depending on the season. This was more than double the emissions produced by conducting the same experiment in Norway, which gets most of its electricity from hydropower, or in France, which relies mostly on nuclear power.
The time of day when the experiments take place is also important. For example, training the AI in Washington during the night, when the state’s electricity comes only from hydro, resulted in lower emissions than during the day, when power also comes from gas stations, said Dodge, who presented the results at the Association for Computing Machinery’s Fairness, Accountability and Transparency conference in Seoul last month.The AI models also varied wildly in their emissions. DenseNet’s image classifier produced the same CO2 emissions as charging a cell phone, while training a medium-sized version of the language model known as a transformer (which is much smaller than the popular GPT-3 language model, created by research firm OpenAI in San Francisco, California) produced about the same emissions. what a typical American household produces in a year. Furthermore, the team only completed 13% of the transformer training process; its full training would produce emissions “on the order of burning an entire railroad car full of coal,” says Dodge.
The emissions figures are also understated, he adds, because they don’t include factors like the power used for data center overhead or the emissions that go into building the necessary hardware. Ideally, the numbers would also include error bars that account for significant baseline uncertainties in the grid’s emissions at any given time, Dontisays.All other factors being equal, Dodge hopes the study will help scientists choose which data center to use for experiments to minimize emissions. “This decision turns out to be one of the most impressive things anyone can do” in the discipline, he says. As a result of the work, Microsoft is now making information about the electricity consumption of its hardware available to researchers using its Azure service.
Chris Preist of the University of Bristol in the UK, who studies the impact of digital technologies on environmental sustainability, says the responsibility for minimizing emissions should lie with the cloud provider rather than the researcher. Providers could ensure that data centers with the lowest carbon intensity are the most used at any given time, he says. They could also adopt flexible strategies that allow machine learning runs to start and stop at times that reduce emissions, Donti adds. Dodge says that the tech companies running the biggest experiments should bear the biggest responsibility for being transparent about emissions and trying to minimize or offset them. Machine learning isn’t always bad for the environment, he points out. It can help design effective materials, model the climate, and monitor deforestation and endangered species. However, the growing carbon footprint of artificial intelligence is becoming a major cause for concern among some scientists. While some research groups are working to track carbon emissions, transparency “has yet to grow into something that’s a community norm,” says Dodge. ,” he says
Source Reference: Elizabeth Gibney, How to shrink AI’s ballooning carbon footprint, nature News (2022), https://www.nature.com/articles/d41586-022-01983-7
