AI training costs continue to fall

The high cost of AI training is a significant barrier to AI adoption, preventing many companies from adopting AI technology. According to a 2017 Forrester Consulting report, 48% of companies cited high technology costs as one of the main reasons for not implementing AI-based solutions.

However, recent developments have shown that the cost of AI training is falling rapidly, and this trend is expected to continue in the future. According to the ARK Invest Big Ideas 2023 report, the cost of training a large language model such as GPT-3 level performance has dropped dramatically, from $4.6 million in 2020 to $450,000 in 2022, a 70% annual decline.

Let’s examine this downward trend in AI training costs and discuss the factors contributing to this decline.

How has the cost of AI training changed over time?

According to a recent ARK Invest 2020 study, the cost of training deep learning models improves 50 times faster than Moore’s Law. In fact, the costs associated with running an AI inference system have dropped dramatically to near-negligible levels for many use cases.

Furthermore, training costs have decreased tenfold annually over the past few years. For example, in 2017, training an image classifier like ResNet-50 on the public cloud cost about $1,000, but by 2019, the cost had dropped significantly to $10.

These findings align with a 2020 report by OpenAI, which found that since 2012, the computing power needed to train an AI model has halved every 16 months.

Furthermore, the ARK report highlights the decreasing costs of AI training. The report predicts that the cost of training a GPT-3 level model will drop to $30 by 2030, down from $450,000 in 2022.

Factors Contributing to Lower AI Training Costs

Training AI models is becoming cheaper and easier as AI technologies continue to improve, making them more accessible to a wider range of businesses. Several factors, including hardware and software costs and cloud-based AI, have contributed to the decline in AI training costs.

Let’s explore these factors below.

1. Equipment

AI requires specialized high-end, expensive hardware to process large volumes of data and calculations. Organizations such as NVIDIA, IBM, and Google provide GPUs and TPUs to run high-performance computing (HPC) workloads. High hardware costs make large-scale democratization of AI difficult.

However, as technology advances, hardware costs come down. According to the ARK Invest 2023 report, Wright’s Law predicts that the cost of producing an AI-related compute unit (RCU), i.e. the cost of AI training hardware, should decrease by 57% per year, leading to the cost of AI training to a 70% reduction by 2030. as shown in the graph below.

2. Software

AI software training costs can be reduced by 47% annually due to increased efficiency and scalability. Software frameworks such as TensorFlow and PyTorch allow developers to train complex deep learning models on distributed systems with high efficiency, saving time and resources.

Furthermore, large pre-trained models such as Inceptionv3 or ResNet and transfer learning techniques also help reduce costs by allowing developers to fine-tune existing models rather than train them from scratch.

3. Cloud-based artificial intelligence

Cloud intelligence-based learning reduces costs by providing scalable computing resources on demand. With a pay-as-you-go model, businesses pay only for their computing resources. Also, cloud providers offer pre-built AI services that accelerate AI learning.

For example, Azure Machine Learning is a cloud-based service for predictive analytics that enables rapid model development and implementation. It offers flexible computing resources and memory. Users can quickly scale up to thousands of GPUs to increase their computing performance. It allows users to work through their web browser in preconfigured AI environments, eliminating installation and setup costs.

The impact of reducing AI training costs

The reduction in artificial intelligence training costs has a significant impact on various industries and sectors, leading to improved innovation and competitiveness.

Let’s discuss some of them below.

1. Mass adoption of sophisticated AI chatbots

AI chatbots are on the rise due to falling AI costs. Especially since the development of OpenAI’s ChatGPT and GPT-4 (Generative Pre-trained Transformer), there has been a noticeable increase in the number of companies looking to develop AI chatbots with similar or better capabilities.

For example, five days after its launch in November 2022, ChatGPT amassed 1 million users. While the cost to run the model at scale today is roughly $0.01 per query, Wright’s Law predicts that by 2030 chatbot applications like ChatGPT will be running at mass scale, much cheaper (roughly $650 to run a billion queries for): with the potential to process 8.5 billion searches per day, equivalent to Google Search.

2. Increased use of Generative AI

Falling AI training costs have led to an increase in the development and implementation of generative AI technologies. The year 2022 will see significant growth in the use of generative AI due to the introduction of innovative generative AI tools such as DALL-E 2, Meta Make-A-Video, and Stable Diffusion. In 2023, we have already witnessed a breakthrough model in the form of GPT-4.

In addition to generating images and text, generative AI helps developers write code. Programs like GitHub Copilot can help get the coding task done in half the time.

3. Better use of training data

Reducing AI training costs is expected to allow better use of data for machine learning training. For example, the ARK Invest 2023 report suggests that by 2030, the cost of training a model with 57 times more parameters and 720 times more symbols than GPT-3 (parameter 175B) is projected to drop from $17 billion to $600,000.

Data availability and quality will be a major limiting factor in developing advanced machine learning models in this low-cost computing world. However, training models will develop the capacity to process about 162 trillion words, or 216 trillion symbols.

The future of AI looks very promising. To learn more about the latest trends and research in artificial intelligence, visit Unite.ai.