Before the Spring Festival, the big model ChatGPT launched by the AI organization OpenAI caused a great sensation in the domestic industry.
An AI practitioner told Digital Intelligence Frontline that there is a wave of AI every five or six years. The last wave of AlphaGO shocked everyone. This wave is ChatGPT. But these two times, everyone’s mentality is quite different. When Google’s AI chess defeated the world champion of Go, everyone watched the news, but this time, many people experienced it from the perspective of consumers. In a month, 1 million users around the world are using and experiencing it, which is a very disruptive experience. This is also the first large-scale autobiography of AI.
Digital Intelligence Frontline learned that in addition to marveling at the amazing experience of ChatGPT, the domestic industry is also rapidly reflecting and acting: where is our gap? How will we face up to and improve the deficiencies? What are the opportunities for Chinese practitioners?
An employee told Digital Intelligence Frontline that after the emergence of ChatGPT, the industry has become lively and dynamic again, and the situation of the past few years has been swept away.
01
How big is the gap between China and the United States
After the launch of ChatGPT, a senior Baidu person told Digital Intelligence Frontline that he was “not interested” in talking about ChatGPT, and there were five flavors in his words. A founder of an AI enterprise said that in the face of ChatGPT’s amazing performance, his heart was itchy and confused, and he lost sleep. He admitted that there was still a long way to go from the scale of the model to the effect.
Someone uses the same question to ask the big model of a domestic manufacturer and ChatGPT at the same time. ChatGPT is far more logical and complete than the domestic big model in terms of the answer. The answer of the domestic big model has an obvious sense of patchwork, mixed with a lot of nonsense beyond the theme. In addition, ChatGPT also leads the way in response speed.
Le Cheng, CEO of Tekan Technology, who is engaged in the research and development of digital people, believes that there is no big model that can compete with ChatGPT in the world at present, and the industry consensus is that the gap is more than two years. In China, it is more important not to talk about overtaking at corners, but to catch up early.
Although some senior AI experts believe that China and the United States are “equal” in the technology involved in ChatGPT, Liu Qun, the chief scientist of voice semantics at Huawei Noah Ark Laboratory, admitted in the discussion of Huang Danian’s Tea House that China still has a gap in technology. One of them is the gap of the basic model itself. Although we have trained many trillions of models or hundreds of billions of models, the adequacy of training is far from enough. “I estimate that up to now, no model can eat as much data as GPT.”
Huang Minlie, associate professor of the Department of Computer Science and Technology of Tsinghua University, said that after GPT-3, all OpenAI models were not open source, but it provided API calls. In this process, it has done one thing: it has built a flywheel between the real user call and the model iteration. It attaches great importance to the call of real world data and the iteration of these data to the model. Of course, in the process, it has also supported a large group of American startup companies and established an ecosystem.
“Look at our large model research in China. Company A has trained one, and Company B has also trained one, and it will be over after an advertisement. The model is open source, and you like to use it or not. At least we haven’t seen a good company to turn the flywheel of data and model completely. So I think this is the difficulty for us to catch up with and surpass ChatGPT.” An insider said frankly.
In addition, people in the industry have mentioned the problem of computing power. Due to problems such as GPU chips, domestic computing power has been blocked to some extent. Even domestic leading companies have a significant gap in computing power compared with Google and others.
In terms of data quality, the Chinese data quality of the whole Internet still lags behind that of English. “We may want to find ways to complement data between Chinese and English languages,” said an insider.
In addition, almost all the interviewees mentioned the pure innovation spirit and long-term spirit embodied by OpenAI, an artificial intelligence organization. “In fact, from the perspective of principles and methods, what they do is understood by the industry, but it is not said that what the United States can do and what we can’t do.” Liang Jiaen, chairman of Yunzhisheng, said to Zhiqian. But like OpenAI and DeepMind, they may be the only two institutions in the industry, and they are consistent in innovation, investment, determination, and top talent reserve. “What we see is success, but there may have been many failed attempts.”
Some senior AI practitioners believe that OpenAI has made a very firm investment in the stage where there is no prospect and no obvious effect. On the contrary, China tends to follow quickly after technology breakthrough. “The first step for everyone in China is to think about how we can use it now, but when we can’t use it, people are investing in it for a long time.”
“This is really worth learning from. We really need to have enough money. With such a group of warm-blooded talents, we can continue to accumulate strength in one direction. I think this is a very necessary condition.” Huang Minlie said.
Recently, the industry is also discussing whether Chinese enterprises can surpass it. Liu Jie, president of Rong Lianyun AI Research Institute, told Digital Intelligence Frontline that there are opportunities to transcend around business, especially domestic scenarios. It is also the consensus of the industry to begin to surpass in local applications.
02
What is the inspiration for China’s AI industry
ChatGPT is a big model. Before its launch, there were actually many large models at home and abroad. Compared with other large models, it has made unexpected breakthroughs in the industry and also brought inspiration to the domestic industry.
First of all, ChatGPT has a very powerful technology base, that is, the InstructGPT model. However, when the paper on this model was just published, it did not cause a great response. Everyone thought it was just a paper on OpenAI. Liang Jiaen also told Digital Intelligence Frontline that GPT and BERT model routes have been competing before. In 2018, BERT model won first, but the GPT model route has not been abandoned. The model parameters and data scale have become larger and larger. Finally, combined with artificial feedback enhanced learning (RLHF), a major breakthrough has been made in ChatGPT, which has surpassed the BERT model route in effect. Therefore, the companies in the industry are paying more attention to the GPT model route, and the technology route alternates with competition, which is the normal in the industry.
Secondly, ChatGPT introduced reinforcement learning mechanism. Rong Lianyun and Liu Jie told Digital Intelligence Frontline that ChatGPT not only uses the data without manual annotation to learn like the previous big model, but also introduces the data with manual annotation in the new version, and optimizes it pertinently through human feedback. “This is an important development of ChatGPT and gives us great inspiration.”
“This is its core point.” Zhao Shiqi, president of Huawei Terminal Cloud Service Search and Map BU, said, “Today, our model is getting bigger and bigger. It is like a force of famine. Some people will fear whether it will control human beings in the future. But after introducing this reinforcement learning mechanism, it is equivalent to giving a guide to the force of famine, allowing the output of the big model to move in a controllable direction, and producing results that meet expectations.” For example, If you ask him some sensitive topics about ethics and safety, he can answer them very well.
The difficulty here is how to establish a reinforcement learning mechanism. Previously, in the AI game of Go, reinforcement learning was to use victory and defeat as feedback. However, there is no clear feedback mechanism for such an open system as ChatGPT. Huang Minlie said that in the past, people have also tried to strengthen the learning mechanism, but many have not been successful. ChatGPT has made a breakthrough in this matter.
Third, it pays great attention to data quality and diversity. OpenAI employs a data team of tens of people. In fact, the amount of data for ChatGPT reinforcement learning is not large, but it has exquisite design for data diversity and annotation system, which makes data play a powerful role. The industry believes that this is indeed something we can learn from.
Of course, ChatGPT also has obvious shortcomings. It is generally acknowledged that it is good at serious nonsense. Liu Jie told Digital Intelligence Frontline that ChatGPT is a black-box computing, and there are certain limitations in the credibility and controllability of content at present. “We should give it enough correct knowledge, introduce knowledge management and information injection technologies such as knowledge map, and also limit its data scope and application scenarios to make its generated content more reliable, which is what we are doing.”
The change of ChatGPT in AI’s technical route and training method makes people pay attention to the changes it brings to the industry. In particular, Le Cheng, CEO of Science and Technology, predicted that there would be a huge difference between using large models in many single-point links. Enterprises that do not use large models and only make products based on their previous generation of small models will not be able to compete with companies that have applied large models and also made business verticalization.
A number of entrepreneurs said that in the past two years, the entrepreneurial market was depressed, and everyone was exploring what technology could significantly improve productivity. “ChatGPT actually provides a new paradigm. The state of despondency and hopelessness may become a thing of the past. It is entirely possible to have several companies with the level of 100 billion in this field,” said Le Cheng.
03
Actions of Chinese enterprises
After the launch of ChatGPT, many people are talking about the anxiety of Google and Baidu. However, most Chinese insiders believe that ChatGPT is still an exploration of AI technology paradigm, and it cannot replace search. ChatGPT has a big disadvantage at present that it can not obtain internet information in real time. Because it is only an end-to-end generation model and can construct false answers by itself, these are all obstacles to its replacement of search. At the current cost of a few cents per strip, it will make commercial search engine companies unable to make ends meet.
As a supplement to the search engine, it has opportunities, because the search engine also stresses “what you ask is what you answer”, but there is still a development process.
Liu Jie believes that the main industrialization path of ChatGPT is mainly at the C end. ChatGPT may focus on some open, creative and universal tasks due to the creativity of large models and the ability to understand long contexts.
However, Chinese enterprises have begun to explore the industry market. For example, in the field of intelligent customer service, Wei Jiaxing, CEO of Cloud Bat Intelligence, told Digital Frontline that last month they tried to introduce ChatGPT to do the Demo test of outbound calls in some scenarios, and called ChatGPT to reply to customers’ questions.
“The core point of applying this technology in the field of intelligent customer service is how to combine the NLP (natural language processing) capability of the large model with the existing NLP in the enterprise before,” said Wei Jiaxing. For example, the express notification system that helps SF make a return visit is based on several standard actions. Under this closed condition, the existing technology of the enterprise should be given priority to meet the current core needs of customers.
In addition to this requirement, AI tools in intelligent customer service have some defects in generalization and generality. When the corpus information is insufficient, AI cannot respond to the problem. The ChatGPT model can complement this ability. Wei Jiaxing reported that the Demo test of cloud bat intelligent outbound call was effective. In other industries, this technology may still be in the state of entertainment, but in the field of intelligent customer service, ChatGPT has commercial potential.
Ronglianyun, a listed company engaged in communication and digital services, will focus on human-computer intelligent dialogue and do core technology and product research and development, such as intelligent customer service, from 2021. At present, we are developing AI content generation products similar to ChatGPT.
But Liu Jie has different views on the scale of the model. “The advantage of ChatGPT is its large size, but it also brings challenges and limitations when it comes to application landing.” He told the front line of Digital Intelligence, “It is meaningless to talk about big and small without scenarios. It is our goal to train models with appropriate scale in specific application scenarios, under specific constraints and on specific data.”
Liu Jie also said that AI is a product technology with a relatively long chain. If a good feedback mechanism is not established, it is difficult to effectively locate and solve the problems found from the front line in the deployment and operation stage, so it is necessary to let the model continue to grow and optimize, “it is not static, it is not delivered and it will not be concerned.”
Yunzhisheng Liang Jiaen told Digital Intelligence Frontline that they have been closely following the cutting-edge algorithms in the industry and are one of the first teams to apply the BERT and GPT2 model methods to actual business systems, After the technology upgrade, it is expected to bring significant experience improvement.
Like other enterprises, the goal of Yunzhisheng is to get through first, and then land in existing businesses such as IoT intelligent voice interactive dialogue and medical industry applications. Liang Jiaen also mentioned the scale of the model. Considering the cost of commercialization, the parameter quantity of the practical model may eventually be reduced to 1 billion.
Luan Qing, the general manager of Shangtang Digital Entertainment Business Department, told Digital Intelligence Frontline that Shangtang has many years of layout in different fields of AIGC, from text, to pictures, as well as video and animation AIGC. The team has invested in technology and industry for a long time. The team focuses more on video AIGC, and superimposes the generated content similar to GPT developed by Shangtang to create short videos, so that everyone can improve production efficiency in the process of creation, “This is our core point”.
Luan Qing said that ChatGPT is essentially an application based on AI big model. The SenseCoreAI large device based on Shangtang has trained and built a super large base model with more than 30 billion model parameters in the field of large visual model, which can effectively support relevant applications.
Li Zhifei, the founder of Go Out and Ask, told Digital Intelligence Frontline that Go Out and Ask has been doing generative applications since the end of 2019, and has been tracking big models since GPT-3 came out in 2020. At present, one industry application of Fali is copywriting.
A senior developer of artificial intelligence for a game told Digital Intelligence Frontline that this technology can see application prospects in the user interaction and production links of the game. For example, it is called when users interact with NPC (a role). Because of ChatGPT’s excellent understanding of natural language, the openness of interaction between users and NPC has been greatly improved. In addition, in the production process, ChatGPT can be used to generate story lines through keywords, which can provide a reference for planning when the plot moves towards design.
In addition, in the field of digital people, the technology CEO Le Cheng told Digital Intelligence Frontline that the big model has changed the content production and interaction mode of digital people.
Before the introduction of the big model, the speech and action of digital people were basically driven by real people’s behavior. With the big model, the output of content can be completed through the big model. Taking live broadcast with goods as an example, digital people companies first establish a virtual anchor image of digital people for local life, knowledge payment and live broadcast merchants, and then connect the big model to help write the script and feet of digital people live broadcast with goods
