Since ChatGPT , Bard, Claude, Midjourney, and other content generation tools were introduced, people have had high expectations for generative AI. So how do generative AI and traditional AI learn differently?
The Learning Process of Generative AI
1. Data Collection
Like traditional AI, the first step is to collect data. For the GPT model, the data consists of a large amount of text. For example, GPT-4 is trained on several gigabytes of text from the internet.
2. Data Preprocessing
This step involves cleaning the data and converting it into a format that the model can understand. For GPT, this includes tokenization when the text is split into smaller parts (tokens).
3. Creating the Training Model
The model is trained using transformer-based language modeling. The model shows a sequence of tokens and asks it to predict the next token in the sequence. For example, when inputting the phrase “A cat is sitting on…”, the model might need to predict the missing word. The model then makes a prediction and calculates the difference between the predicted and actual words. This difference is used to update the model’s weights and improve its predictions.
4. Backpropagation and Optimization
The loss is used to perform backpropagation. This is a process of calculating the gradient of the loss function with respect to the model parameters. Then, using optimization algorithms like Adam or stochastic gradient descent, the parameters are adjusted to minimize the loss.
5. Repeat
Repeat steps 3 and 4 until the model’s performance stops improving.
6. Fine-tuning Data
After the initial training, the model can be further customized for specific tasks, including further training on smaller, task-specific datasets. For example, if a user wants to use GPT to generate medical text, it can be further configured with a dataset of medical articles.
7. Content Generation
After completing the above six steps, the desired content can be generated.
It is important to note that the foundational large model is the “brain” of generative AI, and the entire emerging value chain will support the training and use of this technology (Figure 1). Dedicated hardware provides the enormous computational power needed to train the models, while cloud platforms enhance the utilization of such hardware. MLOps and model center vendors provide the tools, technologies, and practices that enterprises need to debug and deploy foundational large models into end-user applications.
The following figure describes the value chain of generative AI.
Figure 1 The Value Chain of Generative AI.
The Learning Process of Traditional AI
1. Data Collection
The dataset must meet the AI’s objectives. For example, if teaching AI to identify spam emails, we first need a set of emails labeled as spam or not spam.
2. Data Preprocessing
Data usually needs to be cleaned and formatted before it can be used for learning. This includes removing unnecessary information, handling missing data, or converting text data into numerical formats.
3. Data Splitting
The dataset is usually divided into training and testing sets. The training set is used to train the model, while the testing set is used to evaluate the model’s performance on unseen data.
4. Training the Model
The model makes predictions based on the input and calculates the difference between the predictions and the correct outputs (errors). The model parameters are then adjusted to minimize this error.
5. Model Evaluation
After training the model, it is evaluated using the test data.
6. Setup and Optimization
Based on the model’s performance on the test data, adjustments can be made to the model parameters, use a different model, or collect more data.
From the analysis of the learning processes of generative AI and traditional AI, it can be seen that traditional AI can predict or classify new, unseen data, while in generative AI, we can train a machine learning model that will create outputs similar to its training data. This type of machine learning can handle different types of data, such as numbers, text, images, and audio.
Generative AI is a form of artificial intelligence that can create new things, generating content such as audio, text, code, video, images, and other data. Generative AI models are trained through datasets and can generate new data by studying underlying patterns. For example, when using generative AI to tell a story, we only need to provide a beginning, and generative AI can continue the story. A prominent example of generative AI is the GPT-4 language prediction model. Trained on a vast amount of internet data, they can create text that resembles human-created text, with little difference from text written by humans.
Traditional AI is like a strategic master, capable of making wise decisions based on a set of rules. For example, in a human-computer chess competition, the computer knows all the rules, can predict your moves, and decides its own moves based on pre-determined strategies. It does not invent new ways to play; it simply selects a suitable strategy from the pre-programmed strategies—that’s traditional AI. Other examples of traditional AI include voice assistants like Siri, Alexa, and Netflix, as well as Amazon’s recommendation systems and Google’s search algorithms. Traditional AI must adhere to certain rules and cannot autonomously create new content.
The following table mainly compares generative AI and traditional AI. The key difference between the two lies in their functions and application scenarios. Traditional AI is primarily used for data analysis and prediction, while generative AI goes further by creating entirely new content.
Table 1 Comprehensive Comparison of AI
|
Traditional AI |
Generative AI |
|
|
Main Features |
Performs specific tasks |
Can create new data |
|
Research data and make decisions or predictions |
Creates new original content using raw data |
|
|
Works under a set of predefined rules |
Can generate text, images, music, and code |
|
|
Learning Method |
Supervised learning |
Unsupervised learning |
|
Requires labeled data for training |
Does not require labeled data for training |
|
|
Limitations |
Limited to specific tasks |
Details of generated content are uncontrolled |
|
Cannot innovate original content |
Generated content may lack consistency or accuracy |
|
|
Requires large amounts of labeled data for training |
Requires large amounts of data for training |
|
|
Typical Application Scenarios |
Human-computer chess competition |
OpenAI’s GPT-4 |
|
Spam Sieve for macemail filter |
DeepArtpainting transformation |
|
|
Voice assistants (Siri, Alexa) |
Content creation (stories, art, music) |
|
|
Recommendation systems (Netflix, Amazon) |
DeepFake (AI face swap) |
|
|
Search engines (Google) |
Individual AI responses |
In summary, the main difference between generative AI and traditional AI technology is that generative AI can generate new content, and the generated new content is often presented in “unstructured” forms (such as written text or images) rather than arranged in tabular format.
The underlying technology is a type of artificial neural network known as foundational large models, inspired by the billions of interconnected neurons in the human brain. Artificial neural networks need to be trained through deep learning, where “deep” refers to the number of layers in the neural network. Deep learning technology has driven many new advances in the field of AI. Certain characteristics distinguish foundational large models from previous deep learning models. First, foundational large models can be trained using vast, diverse, and unstructured data. For example, a class of foundational large models known as large language models can be trained on a large amount of publicly available text from the internet covering various topics. Other deep learning models can also handle large amounts of unstructured data, but the datasets used for training are usually more specific. For instance, to train a model to recognize certain objects in photos, a specific set of images would need to be used.
In fact, other deep learning models often can only perform a specific task. In contrast, foundational large models can simultaneously achieve multiple tasks and can also generate content. This capability is learned from the broad training data ingested; for example, by learning patterns and relationships, foundational large models can predict the next word in a sentence. This is why ChatGPT can answer questions on various topics, and DALL-E 2 and Stable Diffusion can generate images based on descriptions.
2023 is referred to as the breakthrough year for generative AI. On August 1, 2023, McKinsey released its latest annual global AI research report, confirming the explosive growth of generative AI. With the advancement of technology, the focus of those interested in AI has shifted from technical personnel to corporate executives. Nearly a quarter of surveyed executives stated that they personally use AI tools in their work; more than a quarter of respondents indicated that organizations using AI have incorporated generative AI into their board agendas. Additionally, 40% of respondents stated that their organizations will increase overall investment in AI due to advancements in generative AI technology.
On June 14, 2023, McKinsey analyzed 850 occupations across 47 countries and regions in its latest report titled “The Economic Potential of Generative Artificial Intelligence,” exploring the impact of the exponential development of AI on the global economy and identifying which industries are most affected and which workers face the greatest risk of unemployment. The survey data covered over 80% of the global labor force.
1. Generative AI is Widely Used
In mid-April 2023, McKinsey conducted a live survey. The results showed that the likelihood of the public using generative AI is increasing, with respondents expecting the new technology to change their industries. Next-generation AI has captured the interest of business people: people from different regions, industries, and levels have been using generative AI in their work for years. Among all respondents, 79% stated that they have encountered generative AI in their work or outside of work, and 22% of respondents indicated that they use it frequently in their own work. Among those using AI, 40% of respondents said their organizations expect to increase overall investment in AI due to generative AI, while 28% indicated that the use of generative AI has been included in their board meeting agenda.
The industries that most frequently use generative AI are marketing and sales, product and service development, and service operations such as customer service and back-end support. Previous research has shown that these three areas account for about 75% of the annual total profits from generative AI applications.
Three-quarters of respondents believe that generative AI will trigger significant or disruptive changes in industry competition within the next three years. Respondents from the technology and financial services industries are the most likely to expect significant changes from AI. Previous research has indicated that industries that rely heavily on intellectual labor may face greater challenges and have the potential for greater profits.
Technology companies expect next-generation AI to have the most significant impact—an increase in value equivalent to 9% of global industry revenue. Knowledge-based industries, such as banking (up to 5%), pharmaceuticals and healthcare (up to 5%), and education (up to 4%) may also have a significant impact. In contrast, manufacturing industries, such as aerospace, automotive, and advanced electronics, may experience less disruptive effects. This sharply contrasts with previous technological waves that had the most significant impact on manufacturing.
2. Generative AI is Efficiently Applied in Leading Enterprises
The survey results indicate that organizations with high AI performance metrics are making the most effort to use AI. These organizations benefiting from AI have been using generative AI in more business areas, particularly in product and service development, as well as risk and supply chain management. Considering all possibilities of AI, including more traditional machine learning, robotic process automation, and chatbots, high-performance AI is also more likely than others to use AI for product and service development, such as optimizing product development cycles, adding new features to existing products, and creating new AI-based products. These organizations also use AI for risk modeling more frequently and apply it in human resources for performance management, organizational design, and optimizing workforce allocation.
3. Generative AI Changes Salary Structures
Salaries related to AI will change, and the impact of AI on the workforce is expected to be significant. Survey results show that the role organizations play in supporting their AI ambitions has changed. In 2022, organizations using AI most commonly hired data engineers, machine learning engineers, and all data professionals, which were positions that all respondents reported typically filling in previous surveys. However, compared to the last survey, the proportion of respondents reporting hiring software engineers related to AI is much smaller. Recently, with the introduction of generative AI, roles in operational engineering have emerged due to the increasing demand for this skill set.
Looking ahead to the next three years, respondents predict that AI adoption will change many workforce roles. Overall, they expect more employees to be retrained rather than laid off. Nearly 25% of respondents indicated that more than 20% of their employees will undergo retraining, while 8% of respondents stated that their workforce will decrease by more than 20%.
High-level AI executors are expected to retrain at a higher level than other organizations. Respondents from these organizations indicated that due to AI adoption, their organizations are expected to lay off more than 30% of their workforce in the next three years.
4. Generative AI Will Continue to Receive High Attention and Maintain Stable Influence
Although the use of generative AI tools is rapidly becoming widespread, survey data does not indicate that these new tools contribute to the overall adoption of AI within organizations. The proportion of organizations implementing AI has remained stable, at least for now. 55% of respondents stated that their organizations have implemented AI. Less than one-third of respondents continue to claim that their organizations have implemented AI in more than one business area, indicating that the scale of AI use remains limited. Product and service development and service operations continue to be the two most common business functions adopting AI, consistent with the results of the previous four surveys. Overall, only 23% of respondents indicated that their organizations had at least 5% of their earnings before interest and taxes (EBIT) related to the use of AI in 2022—this has seen almost no change compared to previous surveys, indicating that organizations have greater potential for benefit.
According to the survey, it seems that very few organizations are fully prepared for the widespread use of generative AI or ready to face the business risks that these tools may bring. Only 21% of respondents indicated that their organizations have developed relevant policies to manage employees’ use of AI technology at work. When asked about the risks of adopting generative AI, very few respondents indicated that their companies are reducing the most commonly mentioned risks. The risk of generative AI is uncertainty. Compared to cybersecurity and regulatory compliance, respondents more frequently mentioned uncertainty, which was the most common risk associated with AI in previous surveys. Only 32% indicated that they are working to eliminate inaccuracies, down from 38% who said they were reducing cybersecurity risks. Interestingly, this figure is far lower than the proportion of respondents reporting a decline in cybersecurity risks associated with AI in 2022 (51%). Overall, as in previous years, most respondents indicated that their organizations have not addressed issues related to AI.
As this is a rapidly evolving new technology, many existing regulatory and protective frameworks have not kept pace with the emergence of generative AI and its applications. A major concern is the ability to identify or verify content created by AI rather than humans. Another issue, known as the “technological singularity,” is that AI will become more intelligent and may surpass human intelligence.
The impact of generative AI is extensive: on one hand, generative AI can help us create countless prototypes in minutes, thereby shortening the time required for the creative development process. In the entertainment industry, it can assist people in creating new music, writing scripts, and even concocting fake news. Generative AI plays a key role in creativity and innovation, bringing revolution in any field related to it. On the other hand, traditional AI can continue to be successfully applied to specific tasks. Traditional AI is the driving force behind most AI applications today, optimizing efficiency across various industries.
Specifically, generative AI may bring the following impacts.
1. Promoting improvements in cybersecurity technology
Generative AI may affect the number of cyberattacks on small and medium-sized enterprises, which are easy targets. Cybercriminals use AI to create new and complex attack tools, including malware, exploits, custom phishing, and other techniques. But on the other hand, generative AI can also be used to provide advanced intelligent security tools to improve attack detection technologies and provide fully automated responses.
2. Driving advancements in related technology industries
On July 20, 2023, Silicon Valley startup Cerebras launched a supercomputer that uses dedicated chips aimed at powering AI products. These chips are the size of dinner plates and are 56 times larger than the chips typically used for AI. Each Cerebras chip possesses the computing power of hundreds of conventional chips. The new supercomputer was proposed against the backdrop of an AI boom, stimulating demand for chips and computational power.
3. Changing software engineering work
4. Helping account managers stay updated on public information and data
5. Reducing customer service time, allowing customer service representatives to focus on more valuable work
6. Accelerating drug discovery
7. In the military, applicable to unmanned autonomous systems, ISR, and command, control, and communication (C3)
It is evident that transformative use cases that bring tangible benefits to work and the workplace already exist for generative AI. From pharmaceuticals and banking to retail, many companies are generating a series of use cases to capture value creation potential.
Although generative AI and traditional AI have different functions, they are not mutually exclusive. Generative AI can work in conjunction with traditional AI to provide more powerful solutions. For example, traditional AI can analyze user behavior data while generative AI can use this analysis to create personalized content.
Both generative AI and traditional AI play important roles in shaping the future of humanity, each offering unique possibilities. Mastering these cutting-edge technologies will be a key factor for companies and individuals to keep pace with the rapidly changing digital world.
Compiled by: Jiang Song
Reviewed by: Hu Zhiqiang
Edited by: Hu Zhiqiang, Jiang Song
Audited by: Zhang Yisheng
Corporate Mission: Enhance national defense with electronic equipment and serve society with technology industry.
Corporate Vision: Build a leading domestic and internationally advanced innovative high-tech group in the field of information systems and intelligent equipment.
Core Values: People-oriented, Responsibility, Teamwork, Progressiveness.
