Who is holding back China's ChatGPT?

ChatGPT has become an undoubted phenomenal product in the global information technology industry.

In the name of "general artificial intelligence", it chats with human beings, helps people draft emails and lawyer letters, answers some esoteric ultimate philosophical questions, writes a usable Python code, and answers some questions that seem to require complex and progressive logic. , Write a movie script based on some character settings, write a beautiful love poem, catch a knife for a college student's thesis homework... It seems that there is no such a versatile AI species in human history. Bill Gates said that the significance of the emergence of ChatGPT is "no less than the birth of the Internet", Microsoft CEO Satya Nadella said that it is comparable to the industrial revolution, artificial intelligence oral enthusiasts once again exclaimed that the "singularity" is coming, ordinary People once again worry that their jobs will be replaced by all-round AI assistants like ChatGPT... From IBM's "Dark Blue", to Google's AlphaGo, and then to OpenAI's ChatGPT, 25 years have passed, AI is constantly evolving, and human beings' daily life for AI Responding without showing any mental maturity, this is really something that makes AI happy.

I have used ChatGPT to do many indescribable things, and found that it is not always handy, but it can give better answers and solutions to some seemingly more difficult problems. For example, if you ask it whether BYD can beat Tesla, it may give some clear and unremarkable arguments with many factual fallacies and no personality at the same time; but if you ask it how autonomous driving will change a car For industrial design, it can give a discussion full of imagination from the inside to the outside in terms of chassis innovation, interior changes, digital entertainment, and appearance breakthroughs. On the whole, ChatGPT is quite imperfect, especially in terms of delivering convincing accuracy, but it often surprises humans in areas such as providing structured information discourse, opening imagination and liberating creativity. You can't tell what specific use it has for you, but it can help you achieve and complete some trivial, redundant and even creative things.

It is such a seemingly useless but useful ChatGPT, which seems to be useful but useless, that has driven its parent company OpenAI to receive an additional cumulative investment of more than 10 billion US dollars from Microsoft. It took two days to break through 1 million users. Facebook once spent 305 days; it took two months for it to reach 100 million users, even TikTok took nine months - remember, unlike Facebook and TikTok, ChatGPT is not yet a standalone consumer-grade Internet product, it is still only a A large neural network with 175 billion parameters using the GPT-3 natural language model - when it is preferentially licensed to Microsoft to integrate office software and search engine services such as Office and Bing, it really becomes a "available" The product.

Source: OpenAI official website

But that's enough to make China's AI unicorns jealous.

With 500 employees, the company's overall valuation is close to 30 billion US dollars. This is OpenAI; with thousands of people, the company's valuation/market value is at best 1-2 billion US dollars. This is a number of AI "small giants" in China.

Because of the huge gap between human efficiency and value, and because of ChatGPT's sudden influence on the real society of all human beings, the birth of ChatGPT has greatly stimulated the field of artificial intelligence in China. Many people jumped out again, feeling that the gap between China and the United States in artificial intelligence has further widened, and China has a long way to go to catch up with this wave. There are also some people who are keen to discuss why China does not have its own ChatGPT, and the conclusion is still that "China lacks the soil for innovation" and "Chinese Internet companies are engaged in live broadcasting and grocery shopping", which are both irresponsible and ignorant of the facts.

Chinese Internet companies are not all engaged in live broadcasting and grocery shopping, they are engaged in semiconductor development, AI model research and autonomous driving; American Internet companies are also very popular in live broadcasting, grocery shopping, especially Internet finance. Use a flashlight and a magnifying glass to find your own problems, try to beautify your opponent, use concealer to cover up your opponent's problems, and blame the cause on simple and crude reasons. You can shut your mouth. This is not to reflect on why China can't be the first to create itself The correct pose for a generative artificial intelligence model.

As the Chinese Internet company with the most accumulation in the field of artificial intelligence and natural semantic processing, Baidu has been working on its own deep learning model "Paddle Paddle" for the past five years, and even used its own general-purpose AI chip "Kunlun Core" to train Own models - they are the basic environment and premise for Baidu to train its own "ChatGPT". Alibaba, ByteDance and Didi also have natural semantic training models based on their own needs. It can be said that in terms of training complex natural semantic models with tens of billions of parameters, Chinese companies and research institutions are not weak, and their starting point is not lower than that of their American counterparts—at least around 2016. In recent years, the gap between the Chinese and American artificial intelligence circles in the field of large-scale models is not a matter of awareness, starting point, and ability, but a matter of roads and methods.

The gap between China and the United States in the field of ChatGPT-like human-computer dialogue models is not caused by so-called regulation. If you have had frank exchanges with ChatGPT on some of the richer religious, cultural, ethnic, and geopolitical issues, you will realize that it hides certain stances behind its seemingly refusal and prudent discussion of these issues , which subtly coincides with the mainstream values generally recognized in American society. It can be said that for any complex model of natural semantics, the process of model construction, corpus collection, training, and parameter adjustment is a process of "content review" based on a specific value system, and there is a need to maintain its value system. self-awareness. It is not a question of whether we should "generate" China's value position in the natural semantic model, but how it should be generated in order to truly check and balance the worldview and cultural hegemony that English dominates the global Internet corpus, and strengthen the benchmark for Chinese language understanding. The weight of the global natural semantic processing system provides cultural diversity for the development of artificial intelligence and human-computer dialogue in the world.

I also seriously disagree with the statement that the quality of Chinese Internet information is too bad that the source of the Chinese ChatGPT model corpus is "polluted". This is also a lazy and smart judgment. Because of the total amount of information on the Internet, English content is undoubtedly the most in the world, and the extreme content of worrying quality is also the most, all of which will affect the process and results of natural semantic model training. In the early training, ChatGPT gave priority to the highly praised content on the social forum Reddit with high content quality, which has a specific corpus selection tendency. If China gives priority to knowledge communities such as Zhihu and Dede, and mainstream media give priority to the corpus of semantic models, there will be no problem of corpus contamination. Not to mention the foreign language proficiency and reading breadth of most people who hold that "the quality of Chinese content is low" is not enough to support their conclusions.

But in any case, the birth of ChatGPT is indeed a kind of stimulation and a conceptual challenge for me, who has been calling for "farewell to Silicon Valley worship" for many years.

This is not because I think the gap between China and the United States in the field of artificial intelligence has widened, but because a general artificial intelligence human-computer dialogue model such as ChatGPT is a real possibility from all human beings—not a specific field and From the perspective of the industry, it is a tool to promote social production collaboration and civilization process. Its significance is greater than the emergence of the mobile Internet, comparable to the birth of email and search engines. As a big artificial intelligence country, China is no longer a country with a poor information technology industry when emails and search engines were born. However, we have not allowed this kind of general artificial intelligence innovation that can affect the progress of human civilization to happen first in China. Train a model whose basic corpus is constructed from Chinese culture and value system.

What's more, ChatGPT's model training method largely relies on the parameter upgrade of "strengthening miracles", repeated training, and continuous iterative optimization of the model based on the feedback of generated content-this was originally the working method that the Chinese team was best at. When an American start-up company uses the money raised from Microsoft to invest a huge amount of computing power at any cost, hire a large number of data workers in Africa and the Middle East for information labeling, and use the most efficient iterations to conduct self-developed semantic processing large models with giants like Google You still have a very unreal feeling when it comes to the "arms race" - whether this is a San Francisco company or a Shenzhen company.

A natural semantic processing model like ChatGPT should have been born in China but it was not born in China. The reason has to start with China's technology companies engaged in artificial intelligence—no matter what the giants or startups are doing these years.

A problem that many people may never realize is that a super-large-scale general-purpose natural semantic processing model like ChatGPT is most likely to produce miracles when built by an AI startup company, and better results are usually not achieved within a technology giant. This is why Google's LaMDA dialogue application model and Bard, which has recently rushed into battle, have not shined, and it is also the challenge that Baidu will inevitably face next.

Why? The first is because general natural semantic processing modeling is too expensive. In fact, burning money is usually not the skill of big companies, but the privilege of startups. Tech giants are almost all listed companies. The investment of tens of billions of dollars has been invested in a matter that will not see a return for a long time. The pressure on the chief financial officer in the face of the board of directors and shareholder meetings is very high, and often Being punished by the stock price, this has caused big companies to dare not take big risks, and there will be no big iterations without big risks. What is "Making Miracles Vigorously"? It is to spend a lot of money and make great efforts first, and then pray for miracles to happen, instead of acquiescing that a miracle must happen, and then decide to spend money and make great efforts.

Unfortunately, big companies can only be the latter. This is why even Microsoft, which has benefited a lot from ChatGPT, only dared to start from $1 billion at the beginning, which lasted four years, until this year's $10 billion, and continued to invest one by one to support OpenAI in Microsoft. "In vitro", the GPT model has been trained for many years. The equity acquired by Microsoft through investing in OpenAI enjoys the priority of integrating ChatGPT model capabilities into its Office and search engines. It may not be easy to say whether it will eat OpenAI in the future, but at least the market value is nearly one trillion US dollars, a year Microsoft, with an income of tens of billions of dollars, would never dare to "do miracles vigorously" from the beginning and train this model on its own.

Second, because people are less tolerant of tech giants engaging in innovation, and more tolerant of mistakes and deviations in startups. In order to cope with the pressure of ChatGPT, Google hastily launched the human-computer dialogue test version Bard. It was found that some dialogues had basic factual errors, so it was infinitely magnified, and the market value evaporated hundreds of billions of dollars overnight. In fact, it's not that Google doesn't know this. If it wasn't pushed into a hurry, it wouldn't be so rash. The LaMDA model announced by Google in 2021 has significantly higher parameter levels and information search capabilities than the GPT-3 trained by OpenAI at that time. The decline in stock prices.

What Google cares about, OpenAI doesn't care about. From the first day of ChatGPT's release, it has publicly stated that it has no information retrieval capabilities, and its corpus is only up to December 2021. It cannot answer many questions about value and moral judgments, and often makes factual mistakes. The testers tolerantly accepted ChatGPT's self-"bad performance", and they were all satisfied with its information association, emotional expression, logical structure, and thinking coherence in the fields of programming, literary creation, formatted writing, and medical consultation. Marvel at the power of the series, lightly brushing off the mistakes it made.

In March 2019, after the unprecedented success of the GPT-2 model, the four-year-old OpenAI decided to transform from a non-profit foundation to a commercial company. After all, no foundation can stand its chief scientist’s annual salary of $1.5 million. In May 2019, Sam Altman (Sam Altman) became the CEO of OpenAI. Then, OpenAI received a $1 billion investment from Microsoft. In May 2020, the GPT-3 model launched by OpenAI has parameters that have risen sharply from 1.5 billion in GPT-2 to 175 billion, forming an unprecedentedly powerful automatic learning system.

It can be seen that an artificial intelligence start-up company that was born with a golden spoon in its mouth, raised a huge amount of money, and is bundled with giant business, is engaged in the construction and development of general-purpose artificial intelligence natural semantic models, and invests in model training regardless of cost, which is the most ideal state. The imaginative and commercial returns that come with the most powerful models are enough to spur Microsoft and other investors.

So, why doesn't this logic work in China? Did China ever have a powerful general-purpose natural semantic artificial intelligence model, even if it was just a prototype?

To answer this question, look at when Microsoft first invested in OpenAI: July 2019. Four months after Microsoft’s bet on OpenAI’s GPT model, that is, in November 2019, Shen Xiangyang, a computer scientist from Hong Kong, China, who is in charge of Bing’s search business and is also the top person in charge of Microsoft’s artificial intelligence, announced his departure. Worked for Microsoft for more than 20 years. And Shen Xiangyang's last contribution to Microsoft's general artificial intelligence model is the chat robot Xiaobing developed by Microsoft Asia Internet Engineering Institute in 2014.

In July 2020, Xiaoice became independent from Microsoft and became a Chinese artificial intelligence start-up company. Shen Xiangyang served as the chairman, and Li Di, the former executive vice president of Microsoft Asia Internet Engineering Academy, served as the CEO. When Xiaoice became independent, it had developed to more than the sixth generation. Its product forms involved conversational artificial intelligence robots, intelligent voice assistants, content providers created by artificial intelligence, and a series of vertical field solutions. Xiaoice once sparked public discussion, in addition to being a chat robot full of emotions and female sexuality, it also has its amazing performance in the field of Chinese poetry creation-she published a collection of poems "Sunshine Lost the Glass Window", which has gained a lot of attention. A lot of praise, and more controversy.

There is no doubt that a XiaoIce robot that can write poems and carry out simple emotional and commonsense-based conversations was the best-performing conversational general artificial intelligence model in the world a few years ago.

It is impossible for the team led by Shen Xiangyang to understand search, let alone artificial intelligence. And Shen Xiangyang's departure from Microsoft and Xiaobing's "independence", coupled with Microsoft CEO Nadella's investment and cooperation with OpenAI, is actually the top artificial intelligence trader in China and the United States. Formally parted ways.

So, does Xiaobing still write poetry today? what is it doing

In the past two years, Xiaobing has long stopped writing poems. It's busy commercializing. It established a game studio to provide NPC scripted dialogue content for games; it cooperated with the Winter Olympics to provide a visual scoring system for freestyle skiing aerial skills; it provided Wind Information with artificially generated text summaries of listed company announcements; Vanke and other companies have customized virtual digital humans for customer service... It is working hard to become an artificial intelligence solution company that "empowers" all walks of life and at the same time allows itself to generate blood and make money.

In a word, in the past, the artificial intelligence team that represented the higher level of the general natural semantic artificial intelligence model, and the Chinese supported the entire pattern, has now become a mixture of generative artificial intelligence and decision-making artificial intelligence, providing specific solutions for specific scenarios Solution artificial intelligence provider.

You can’t say that this is Xiaoice’s “fallen”, after all, it has only raised hundreds of millions of yuan from the capital market. According to ChatGPT's model training method, the money will be spent in one day. Without Microsoft's protection, Xiaobing has to take care of his own life. However, I have never heard of Baidu, Tencent or ByteDance. I thought about investing in Xiaoice and supporting it to continue to develop a large-scale model of general natural semantic artificial intelligence.

Not just Xiaoice. In the past few years, there have also been other entrepreneurial teams in China that are engaged in automatic modeling of general artificial intelligence and heterogeneous computing, allowing 7-8 kinds of chips at home and abroad to access software through this model, but as long as they use this model to raise funds, they will not be able to do anything. an investor. Chinese investment institutions have never shown interest in general artificial intelligence models, and even a little bit of imagination.

"More than 85% of the investors asked us to introduce the product scenario as soon as they came up. We said that we help the GPU connect with the software ecosystem, and even Nvidia uses our model. Investors said that this is not a scenario. We said that we also have customers, satellite, Research on docks, smart cities and smart industries, they say you are doing too loosely, so we don’t vote.” This is what I have heard from entrepreneurs who are doing general artificial intelligence models.

As we all know, VCs in China like to "educate" entrepreneurs the most, and of course educate scientists who are engaged in artificial intelligence entrepreneurship. "You have to have some data in this industry", this is their favorite sentence to educate AI entrepreneurs.

There is data in a certain industry, and it is necessary to focus on providing solutions in a certain segmented field. This is the mindset of most VCs and PEs in China who claim to invest in artificial intelligence. Then we look at "how big is the scene", the scene of the security camera is big enough, so the valuation model becomes the size of China, how many cameras can be installed? How much is each camera? How big is the total camera plate? Well, the plate is big enough, and we voted in the subdivision of the camera. Let’s look at port smart logistics again. How many ports are there in China? How many are deep water ports? How much can each port terminal pay for AI solutions? It turned out that we paid such a small amount of money. It seems that the scene of "port" is not big enough, so we will not vote. AI virtual digital human as customer service? It can be linked to the metaverse, it has a story and imagination, well, we can give it a try.

So, what you see is that China's artificial intelligence "four tigers" are basically engaged in the business of cameras and face recognition, and they have all become AI project implementation and integrators. The business model is the same as that of Neusoft 30 years ago. And iSoftStone, they are living in a difficult situation, with huge losses, and they have to support the ranking of China's artificial intelligence industry, and support the valuation and imagination of the field of artificial intelligence.

For quite a long period of time, few investors in the field of artificial intelligence sincerely believed that a general model could be reused in various industries. Occasionally, there are a few who are a bit patient and interested in the general model, and they are basically RMB funds. The US dollar fund is really not interested in the Chinese team's attempt to develop a general model. Do you think that by comparing the difficulty and level of model training of companies like OpenAI and Google, they feel that there is a gap between the Chinese team in this matter? Then you really think too much. They know the time when the GPT model development is going on, that is, the last two months.

Those first-line investment managers who boasted that "In my eyes, SenseTime and Megvii are selling security cameras", those first-line investment partners who proudly told entrepreneurs that "your model is not a scene", let alone those who have almost Instead of investing in artificial intelligence, the partner of the US dollar investment fund that has been tinkering with Chinese entrepreneurs to "go overseas" to engage in cryptocurrency for so many years has suddenly changed today and declared that he will support entrepreneurs to engage in "China's ChatGPT". Then you can think about it, their vows and smugness contain some understanding and sincerity of the general artificial intelligence model, and some are speculation and calculation.

You can even think about it, the training of a super natural semantic model may cost tens of millions or even hundreds of millions of RMB a day, not to mention the computing power module that provides large model training - the world's top GPU, because of the unreasonable embargo of the United States and become increasingly difficult to obtain. With the attitude and behavior style of those investors in the past so many years, they can persist for a few days, how much money are they willing to persuade the investment committee to invest in, or can they help these entrepreneurial teams solve the GPU problem? Maybe one day, maybe half a year later, they will start urging these general model teams to "realize commercialization in subdivided fields" as soon as possible.

With Baidu's insistence on investing in the PaddlePaddle model, it is inevitable that it will practice this model industry from the very beginning, and pursue commercialization in different industries as soon as possible. To a large extent, the training of large models of general artificial intelligence has an "impossible triangle" of massive data, high-quality and creative content output, and industrial application.

To achieve massive data and high-quality creative content output, it is bound to not be able to be quickly applied to the specific implementation of a certain industry - such as ChatGPT.

If you want to create specific industrial landing scenarios in the largest mass data created by humans on the Internet, you will definitely not be able to provide the highest quality results, because there must be conflicts between content generation based on mass data and precise decision-making systems-this is actually a waste.

If you want to achieve high-quality content output to assist accurate decision-making in industrial landing scenarios, you must sacrifice the most massive data, and the data owned by most accurate industrial scenarios cannot support real large-scale model training and research. This is the dilemma faced by most of China's "industry segmentation" artificial intelligence solutions today, and it is also the reason why the so-called "industry ChatGPT" is a false proposition that changes the soup without changing the medicine.

Those entrepreneurs and investors who are gearing up to enter the "China's ChatGPT" today, not to mention how much money and how many GPUs you have in your pockets, since you are all on this ship, you feel that you are holding the ticket. Which corner of the "impossible triangle" of general artificial intelligence will you decide to discard? This is a question that needs to be figured out first.

In other words, which investment institution—whether it is a financial investment institution or an investment department of a large company—has the determination to invest in training large-scale natural semantic models for several years and extend the return cycle infinitely? After all, history tells us that this is a group of people who are the least determined and most anxious to find a successor.

China has never lacked outstanding entrepreneurs and scientists, and the field of artificial intelligence is no exception. The level and accumulation of Chinese and American technology companies in the field of artificial intelligence are the closest in the world. At least a few years ago, there was not a big gap between China and the United States in the construction and training of large natural semantic models. However, China does lack some investment institutions and investors who have a broader vision, do not follow others' opinions, and are determined and far-sighted.

People like Shen Xiangyang, Li Di, Ma Weiying, Wang Xiaochuan, and Li Zhifei are quite reliable when they come out to make large-scale generic natural semantic models, but the problem is that they need to change a group of investment institutions and investors who support them. Some investment institutions that are too good at "playing the game" and speculating, and immersed too deeply in the cryptocurrency and other tracks, should be blacklisted.

To be honest, although no serious investment institutions have been looking at general artificial intelligence models for so many years, there are still some institutions that have invested in many artificial intelligence companies with extremely long payback cycles. For example, those VCs who have invested in China's local lidar and autonomous driving solutions have contributed to the establishment of China's brand-new competitiveness in the global auto industry's unprecedented changes in a century. For example, those VCs who have invested in China’s local GPUs—this is destined to be a track full of hardships, facing the ban and suppression of the United States, and the return cycle is extremely long; but these new local GPU players—whether it is Hanbo, Biren Or else, they may provide ammunition for China's general natural semantic processing model in the future. The investors behind them, if one day they really make up their minds and make a move to support China's natural semantic large-scale model project, I may have some different expectations and confidence in them.

It's just that there are not too many investors and investment institutions who don't brag, don't hold back, and don't rush for quick success, but there are too few, but China's natural semantic model construction and training need such investors and investment institutions—whether it is Financial investors, strategic investors, or capital institutions supported by the will of the state.

China must have its own general natural semantic large model, it needs to have the vision of providing Chinese wisdom, Chinese value system and Chinese solutions for global general artificial intelligence, and it needs to avoid the whole process of corpus selection, model construction and training, and parameter adjustment Risks and legal, moral and ethical issues require determination and patience.

In any case, it cannot be speculated.