The true cost of AI innovation

Print Friendly, PDF & Email

In this special guest feature from Scientific Computing World, Tate Cantrell, CTO, Verne Global, comments on the cost of the AI computing revolution.

Tate Cantrell is CTO of Verne Global.

We should all be thankful that the progress of AI is moving at lightning speed. No longer strictly the province of the research lab, instead it is expanding into everyday life. From neural networks that predict fraudulent credit card transactions to AI-powered Google Maps that can ascertain the speed of traffic using anonymized location data from smartphones. But while everyone is looking to AI’s bright future, the rapid growth has the potential to do more harm than good.

AI is data-driven and heavily dependent on computing power. Depending on the complexity of the machine learning or deep learning model at hand, AI can involve a staggering amount of data, requiring massive computational resources. Given the sizable energy requirements of modern tensor processing hardware, this results in enormously high power consumption.

Since 2012, the amount of computing power used for deep learning research has been doubling every 3.4 months, according to OpenAI researchers Dario Amodei and Danny Hernandez.

This equates to an estimated 300,000-fold increase from 2012 to 2018, far outpacing Moore’s Law, which states that the overall processing power for computers will double every two years.

And as the world’s demand for such AI technology continues to grow, so does the AI industry’s energy consumption. In an environmentally hostile chain reaction, rapidly increasing computational needs will unavoidably escalate carbon costs.

Environmentally 
accountable AI

By its nature, deep learning is extremely compute-intensive. Deep learning is based on neural networks that are comprised of multiple layers, with manifold parameters that can number in the billions. The greater the network depth, the greater the compute complexity, which requires high-performance computational power and longer training times. Canadian researchers, Victor Schmidt et al. report that state-of-the-art neural architectures are frequently trained on multiple GPUs for weeks, or even months, to beat existing achievements.

At present, the vast majority of AI research is focused on achieving the highest levels of accuracy, without much concern paid to computational or energy efficiency. In fact, competition for accuracy in the AI community is robust, with numerous leaderboards tracking which AI system is performing a given AI task the best. Regardless of whether the AI leader board is tracking AI programs for image recognition or language comprehension, accuracy is by far the most important metric of success.

But as the world’s attention has shifted to climate change, the field of AI is beginning to take note of its carbon cost. Research done at the Allen Institute for AI by Roy Schwartz et al. raises the question of whether efficiency, alongside accuracy, should become an important factor in AI research, and suggests that AI scientists ought to deliberate if the massive computational power needed for expensive processing of models, colossal amounts of training data, or huge numbers of experiments is justified by the degree of improvement in accuracy.

Research by the University of Massachusetts (Strubell et al., 2019) demonstrates the unsustainable costs of AI. It analyzed the computational requirements for a neural architecture search for machine translation and language modeling. The model ran for a total of 979 million training steps, and took 10 hours to train for 300,000 steps on one TPUv2 core, equating to 274,120 hours on eight P100 GPUs.

The estimated carbon cost of training the model was 626,155 lbs of carbon dioxide emissions, which is comparable to the amount produced by 125 round-trip flights from New York to Beijing. Given the significant impact that ever-expanding AI research could have on the environment, it is critical that the field of AI starts to weigh sustainability against utility.

How organizations can 
power sustainable AI

Information and Communications Technology (ICT) already accounts for approximately four per cent of worldwide carbon emissions, according to The Shift Project research, and its contribution to greenhouse gas emissions is 60 per cent higher than the aviation industry.

As more enterprises and organizations turn to AI and machine learning applications in an effort to drive innovation, there is a corresponding increase in demand for cloud optimized data centre facilities. If Anders Andrae, senior researcher at Huawei, is right in his prediction that by 2025 data centers will account for 33 per cent of global ICT electricity consumption, the sustainability of AI is a conversation that green-minded organizations desperately need to start having.

There are positive steps companies can take to minimize their carbon footprint whilst still accessing cutting-edge supercomputing to drive their innovations. Given that machine learning and deep learning applications consume an enormous amount of energy, companies need to ensure that the data centers housing those applications can efficiently handle the high-density compute involved, at industrial scale. Many corporate data centers simply are not equipped to handle these demands. According to a survey by Science Direct, out of 100 data centers, 61 per cent were operating with systems running at their lowest efficiency.

Likewise, it is crucial that these facilities are powered by renewable energy sources. If these power-hungry AI applications are housed in fossil-fuel-powered facilities, energy efficiency efforts can quickly become voided. Equally, organizations that rely on cloud service providers should verify their provider’s green credentials. If a cloud provider’s data centre is located somewhere like the UK, which is predominantly powered by natural gas, no matter how many green certificates it boasts, at the end of the day, it is still powered by fossil fuel. Despite what the label suggests, these green certificates do not always signify that the energy being used is renewable. Energy can be certified green even if it is not renewable, as these certificates are akin to a carbon offset program.

Data centre location is also a key consideration when it comes to sustainable AI. Cooling the air inside data centers can be expensive and relatively inefficient, and in hotter climates, keeping hardware cool is particularly energy-intensive. Vitally, more than 80 per cent of computer hardware does not need to be located near the end-user in terms of latency or accessibility. Acknowledging this, it is both economically and ecologically sound business practice to house AI servers somewhere with a consistently cool climate.

Tech giants, like Google, are investing in data centers in Nordic countries specifically because of better energy efficiency, as compared to locations in warmer climates. In a conventional data centre, cooling IT equipment constitutes 40 per cent of the total energy consumed. However, in countries like Iceland, which is perennially cool, natural cooling of powerful AI servers minimizes energy usage and results in considerable energy savings.

Further, Iceland’s energy is sourced from 100 per cent renewable geothermal and hydroelectric power, and its national grid is modern and reliable, meaning the AI systems housed there operate more efficiently and deliver cleaner energy. By making smarter choices about where AI compute is located, organizations can make a substantial impact on the sustainability of their AI.

The future of AI needs to be green

The growth and proliferation of AI shows no signs of abating. According to research firm IDC, worldwide spending on AI systems will be close to $98 billion (£75.4 billion) in 2023. If, as the UN has warned, 2020 is the year that the world must act to avoid runaway climate change, the field of AI – and the organizations that drive it – must begin the new decade by tackling the sustainability of AI head-on, concentrating on energy efficiency from research start to enterprise finish.

This story appears here as part of a cross-publishing agreement with Scientific Computing World.

Sign up for our insideHPC Newsletter