ChatGPT is a natural language human-computer interaction application developed by OpenAI in the United States. It has language understanding and generation capabilities close to human level. It is by far the most successful product in the field of artificial intelligence and the fastest growing application in history. ChatGPT relies on large models, big data, and large computing power. Its emergence marks the starting point of general artificial intelligence and the inflection point of strong artificial intelligence. It is a milestone technological progress that will trigger a new round of artificial intelligence revolution.
The domestic artificial intelligence “big model” already has a certain foundation, but there is still a certain gap with ChatGPT, which faces deep constraints such as data, computing power and innovation environment. It is necessary to pay attention to the new round of artificial intelligence revolution triggered by ChatGPT from a strategic height, aim at large models, integrate big data, deploy large computing power, implement inclusive and prudent supervision, leave enough room for the development of new things, and accelerate to seize the commanding heights of future technological competition.
ChatGPT is a milestone and will trigger a new round of artificial intelligence revolution
ChatGPT (Chat Generative Pre-trained Transformer, Chat Generative Pre-trained Transformation Model) is a natural language human-computer interaction application developed by OpenAI Company of the United States. The ability to create content, write codes, etc. enables people to intuitively and truly experience the tremendous changes and efficiency improvements brought about by the advancement of artificial intelligence technology. The number of users exceeded 1 million within 5 days of going online, and the number of active users exceeded 100 million within two months. The most successful product in the smartphone space and the fastest growing app in history.
ChatGPT is an artificial intelligence “nuclear explosion point” with a certain chance of success after a long-term technical reserve and a large amount of resource investment. The development of ChatGPT has gone through three stages (as shown in the figure below). The previous versions of GPT-1 (2018), GPT-2 (2019), and GPT-3 (2020) have invested a lot of resources (including purchasing high-end Performance chips, employing data labeling personnel, occupying computing resources, etc.), the effect is not ideal. Later, after adopting the “human feedback learning based on reinforcement learning” technology, a “butterfly change” occurred, and it quickly became a popular application.
The key to ChatGPT lies in the “three major supports”. One is the “big model”. The full name is “Large Language Model” (Large Language Model), which refers to a natural language processing model with a large number of parameters (currently reaching hundreds of billions) and using a large-scale corpus for training. It is the “soul” of ChatGPT. The second is “big data”. GPT-1 uses about 7000 books to train the language model. GPT-2 collected 40GB of text data from more than 8 million documents on the Reddit platform (the fifth largest website in the United States, which is similar in function to Baidu Tieba in China). GPT-3 uses high-quality text data from many databases such as Wikipedia, and the data volume reaches 45TB, which is 1150 times that of GPT-2. The third is “big computing power”. Taking GPT-3 as an example, its parameters amount to 175 billion. It uses a high-performance network cluster composed of 10,000 Nvidia V100 GPUs. It takes 14.8 days for a single training, and the total computing power consumption is about 3640PF-days (if 1,000 PF-days are performed per second) trillion calculations, takes 3640 days).
ChatGPT marks a milestone technological advancement. One is a revolutionary breakthrough in the most challenging field of natural language processing. Compared with videos, images, voices, etc., the syntax, semantics, and logic of natural language are complex, and there are characteristics such as diversity, polysemy, and ambiguity. Text data is scarce and often presents as unstructured, low-quality data. There are a wide variety of natural language processing tasks, including language translation, question answering systems, text generation, sentiment analysis, and more. Therefore, natural language processing has long been considered the most challenging field of artificial intelligence. ChatGPT not only achieves high-quality natural language understanding and generation, but also enables zero-shot learning and multilingual processing, bringing unprecedented breakthroughs in the field of natural language processing. The second is to mark the starting point of general artificial intelligence. Prior to this, the application of artificial intelligence in different scenarios required the training of different models. However, ChatGPT can use a single large model to complete various tasks such as human-machine dialogue, machine translation, and coding testing. It already has some core technologies and characteristics of general artificial intelligence: it can automatically learn various knowledge and information, and continuously optimize itself; fully It understands and expresses human language fluently, and has strong logical reasoning, realizing machine intelligence with general human intelligence; with certain adaptive and transfer learning capabilities, it can be applied to a variety of application scenarios and tasks. The third represents the inflection point of strong artificial intelligence. ChatGPT proves the learning and evolution capabilities of large models, which will promote the accelerated evolution of strong artificial intelligence (machines with perception and consciousness, real reasoning and problem-solving abilities). At present, the intelligence level of the large model is close to the human level, and some people in the industry even believe that in the future, self-awareness and perception will gradually appear, and then consciousness will appear and surpass human beings.
The evolution of global general artificial intelligence technology is accelerating. ChatGPT involves the “big model” in the “big three” is the core and unique cheats. At present, more and more “big models” hidden behind ChatGPT are coming into people’s field of vision. There has been a technological upsurge in the world from “refining large models” to “refining large models”. OpenAI will continue to promote the evolution of the ChatGPT model. At present, it has released the multi-modal pre-training large model GPT-4, which has achieved several leaps: powerful image recognition capabilities; the upper limit of text input has been raised to 25,000 words; the accuracy of answering questions is obvious Improve; can generate creative text, lyrics, achieve style changes, etc. Google created LaMDA, a large-scale natural language dialogue model with 137 billion parameters. Currently, the launch of the LaMDA-based chat robot Bard is being accelerated, and the company is mobilizing for internal testing. Microsoft and Nvidia have jointly launched the MT-NLG model with 530 billion parameters. Compared with the previous systems of the two companies, the advantage is that it is better at various natural language tasks, such as automatic sentence generation, question and answer, reading and reasoning, word meaning disambiguation wait. Meta Corporation reproduced GPT-3 and made it freely available to all communities.
The artificial intelligence model represented by ChatGPT has penetrated into all walks of life and will trigger a new round of artificial intelligence revolution. In essence, ChatGPT is a “big model” (a probability model with a huge amount of parameters), and its successful practice has fully proved the potential of a large model as a general technology in all aspects of human society. One is to successfully explore the business model of a large model. ChatGPT has been applied to commercial search engines and office software. The Microsoft Bing search engine embedded with GPT-3.5 can better understand and respond to user queries and provide more accurate search results. The Office software embedded with GPT-4 has greatly improved office efficiency. Second, in the short term, large models will replace some jobs in the service industry. ChatGPT can complete various text generation tasks and replace part of the work of administrative personnel, scientific researchers, legal professionals, media practitioners, and customer service personnel. Be able to code, detect security vulnerabilities, and replace some of the work of a software engineer. It can complete the conversion between languages with high quality, replacing part of the work of translators. The third is that with the continuous penetration of large models, people’s production and lifestyle will undergo profound changes. In the near future, large models that are widely developed and applied will perform automated production and intelligent manufacturing tasks with a speed and accuracy exceeding that of humans, empowering various industries such as transportation, medical care, and finance. This will trigger a new round of intelligent revolution represented by strong artificial intelligence and general artificial intelligence, greatly improve production efficiency, and bring about profound changes in the economy, society, and industry.
The current situation and problems of artificial intelligence “big model”
The large model has a certain foundation, but there is still a certain gap with ChatGPT. One is the “Wenxin” large-scale model independently developed by Baidu, with a parameter scale of 260 billion. It has released 11 industry large-scale models in the fields of energy, finance, and manufacturing. The second is the launch of the multi-modal M6 large model with 10 trillion parameters by Alibaba Dharma Academy. The third is the Pangu large model jointly developed by Huawei and Pengcheng Lab. It is the first fully open source 200 billion parameter Chinese pre-training language model. It has outstanding performance in text generation fields such as knowledge question answering, knowledge retrieval, knowledge reasoning, and reading comprehension. Fourth, the Beijing Zhiyuan Artificial Intelligence Research Institute launched Enlightenment 2.0 with 1.75 trillion parameters, which can process Chinese, English and picture data at the same time. Inspur and the Chinese Academy of Sciences have also launched corresponding large models.
From the perspective of technical capabilities, experts judge that the current domestic technology is mainly inferior to ChatGPT in the large model link, including cleaning, labeling, model structure design, and technical accumulation of training and reasoning. Behind ChatGPT is the integration and innovation of multiple technologies such as text/cross-modal large models, multi-round dialogues, and reinforcement learning. However, most domestic technology companies and research institutes focus on vertical applications and lack the ability to integrate and innovate multiple technologies. From the point of view of landing applications, leading domestic companies have indicated that they have carried out relevant technology research and development or some models have entered the internal testing stage, but there is still no large-scale model product that can compete with ChatGPT. In addition, the training cost of large models is relatively high, technology application is faced with R&D investment of 100 million yuan and massive training trials and errors, the investment of domestic enterprises is seriously insufficient, and R&D promotion and industrial implementation lag behind overseas as a whole.
Related Policy Recommendations
The large-scale artificial intelligence model has important strategic significance. It is the commanding height of future technological competition and an important intelligent infrastructure. It is necessary to pay strategic attention to the new round of artificial intelligence revolution triggered by ChatGPT, accelerate the layout and breakthroughs in terms of algorithms, computing power, and data, build an inclusive and innovative regulatory environment, and actively respond to the new round of artificial intelligence technology competition.