Today, the media reported a piece of news: Baidu was the first to discover scaling law, the core theory of large models. The source is a report by the English newspaper South China Morning Post that although the United States has always been regarded as a leader in AI model innovation, the latest discussions show that China may be more advanced in exploring these concepts. The industry realized that Baidu discovered Scaling Law before OpenAI.

Scaling Law is also known as the "law of scaling" and is considered by the industry to be the first principle of large model pre-training. That is, the larger the training data and model parameters, the stronger the model's intelligence. OpenAI published the paper "Scaling Laws for Neural Language Models" in 2020, and since then, this concept has become a cornerstone of AI research.

The launch of the ChatGPT series of large models made OpenAI a blockbuster and became a hot star in the field of AI. In addition, American companies such as Google, Meta, and xAI continued to launch large models with powerful performance, making many Chinese people subconsciously believe that the United States is in the field of AI. significantly ahead in the field. In fact, China's AI technology is not weak and has always been in a state of parallelism. This is why the "Battle of 100 Models" will occur in 2023. The exposure of the past that “Baidu was the first to discover Scaling Law” reflects China’s deep foundation in the field of AI.

Baidu is better than OpenAI Having discovered Scaling Law earlier, many Chinese are both happy and a little confused. After all, OpenAI is at its peak, and any challenge to it requires strong evidence.

According to reports from all parties, this incident has been confirmed. There are mainly three aspects of evidence:

First of all, Dario Amodei, a master in the AI ​​industry and founder of Anthropic, mentioned in a podcast in early November that when he and Andrew Ng were researching AI at Baidu in 2014, he had Discovered the Scaling Law of model development. Dario Amodei worked at Baidu Silicon Valley Lab from November 2014 to October 2015 and served as Vice President of OpenAI Research. He can be called a party involved and his remarks are very weighty.

Secondly, Jack Morris, a Meta researcher and doctoral candidate at Cornell University, publicly supported it on X (Twitter): "The original research of Scaling Law came from Baidu in 2017, not OpenAI in 2020 ". Later, some media people posted "DEEP LEARNING SCALING IS PREDICT" published by Baidu in 2017.ABLE, EMPIRICALLY" paper, which discusses the Scaling phenomenon in machine translation, language modeling and other fields in detail, further proving his remarks.

In addition, there is another important piece of evidence: Gwern Branwen, an AI expert who previously worked at Open AI and participated in the development of important projects such as GPT-3, often mentioned in public that this Baidu paper was indeed used. ignored.

Baidu’s investment and contribution in the field of AI theory can be seen. Since then, Baidu has continued to move forward and took the lead in launching the large language model Wenxin Yiyan in China. The "Generative Artificial Intelligence Development and Application Development Report (2024)" released by the China Internet Network Information Center shows that Wen Xinyiyan has become the most used generative artificial intelligence platform in China.

Baidu can The discovery of Scaling Law ahead of OpenAI is inseparable from its early investment in AI research and recruitment of AI talents, including AI giants from all over the world.

In addition to the above-mentioned Dario Amodei, Baidu also recruited the more well-known top AI scientist in the country, Andrew Ng, the former founder of Google Brain, which also triggered heated discussions in the media that year.

In addition, according to previous media reports, Geoffrey Hinton, the godfather of AI and winner of the 2024 Nobel Prize in Physics, is also among Baidu’s recruiters. In 2012, Robin Li personally sent people to the United States to extend an olive branch to him, which even alerted the two giants in Silicon Valley, Google and Microsoft, to issue invitations. Due to age and other factors, Hinton finally stayed in the United States, and Baidu unfortunately missed out.

Although external knowledge of Baidu and Scaling Law has just been revealed, it has also made the industry realize that Baidu has previously accumulated deep experience in AI technology, including large models. Data shows that Baidu released the first-generation Wenxin large model as early as 2019, almost at the same time as OpenAI. Since 2023, large models have sprung up like mushrooms after a rain, and Baidu has always been in the lead.

For example, the China Software Evaluation Center's "Research Report on the Development of Artificial Intelligence Large Language Model Technology (2024)" shows that Wenxin's large model technology, products, and applications are leading the way; Sullivan's "China's Large Model in 2024" "Capacity Evaluation" shows that Wenxin's large model firmly ranks first among domestic large models, winning 4 first places. Baidu's AI technology has been accumulated for more than 10 years and will be fully developed in 2024.

In fact, not only Baidu and the entire Chinese AI industry are in a highly prosperous development stage.In terms of development status, there is no obvious gap between leading domestic AI companies and overseas leading companies in terms of technological innovation, product innovation or application innovation, and they are generally on the same level.

According to the latest "Global Information Society Blue Book: Global Information Society Development Report (2024)" released by the Shanghai Academy of Social Sciences, China, the United States, and the European Union have demonstrated their performance in artificial intelligence research, innovation, education, and application. The amount is about the same, and artificial intelligence is still the world’s development pattern dominated by China, the United States, and the European Union.

As one of the representative AI companies, Baidu, together with Google and OpenAI, was rated as an "AI-Native Giant" by Sullivan's 2024 Global AI Panoramic Report, and is in a leadership position.

At present, the AI ​​era has arrived, showing great potential to transform the economy and society, and competition in the AI ​​industry has become increasingly fierce. AI technology has become the winner of the technological competition between major powers. The United States continues to resort to outside measures to restrict the development of my country's AI technology. President-elect Trump even plans to appoint an artificial intelligence minister (AI czar) to start an AI arms race.

In this context, Chinese people must have confidence in the development of the domestic AI industry. In this long-distance race, many Chinese companies, including Baidu, are investing intensively in AI technology innovation and have achieved a lot of results, which will promote the development of the AI ​​industry to prosperity and become a booster for high-quality economic transformation and development.