Source: Overseas Unicorn

We have been talking about the second half of AI since 24Q3. Although OpenAI o1 proposed the RL narrative, it did not break the circle for various reasons. DeepSeek R1 solved the RL The puzzle has pushed the entire industry into a new paradigm and truly entered the second half of intelligence.

There are already many discussions on what DeepSeek is and why. The next more valuable discussion is how to play AI race? I summarized the thoughts of the past half month, hoping to become a road map for exploring the second half, and reviewed them every once in a while. There are also a few of the most curious questions listed. Welcome to fill out the questionnaire and exchange ideas together. We will organize a small-scale discussion on the next intelligent breakthrough Aha moment:

•Next intelligent breakthrough Where will Aha moment appear?

•If you have abundant exploration resources, in which direction will you invest your exploration resources?

•For example, the next generation Transformer architecture, breakthroughs in synthetic data, and Online Learning’s more efficient learning methods, what bets do you have?

Did Insight01DeepSeek surpass OpenAI?

DeepSeek surpasses Meta Llama. There is no doubt that, but it is still far from the first-tier players such as OpenAI, Anthropic and Google. For example, Gemini 2.0 Flash costs less than DeepSeek, has strong capabilities, and is fully modal, outside world The ability of the first echelon represented by Gemini 2.0 is underestimated, but it has achieved such explosive and sensational effect without open source.

DeepSeek is very exciting, but it cannot be called paradigm-level innovation. A more accurate statement is to open source the previous OpenAI o1 half-covered paradigm, pushing the entire ecosystem to a very high penetration rate.

From the perspective of first-principles, it is difficult to surpass the first-tier model manufacturers under the Transformer generation architecture. It is difficult to achieve the possibility of overtaking on the curve under the same path. Today I hope someone can explore it. A generation of intelligent architecture and paradigms.

DeepSeek took a year to catch up with OpenAI and Anthropic.

Insight02DeepSeek Has the new paradigm opened?

As mentioned earlier, DeepSeek strictly speaking, there is no new paradigm inventing.

But the importance of DeepSeek is to help RL and test time compute thisThe new paradigm has really emerged. If the initial o1 released by OpenAI was a riddle for the industry, DeepSeek was the first person to solve the puzzle publicly.

Before DeepSeek released R1 and R1-zero, only a few people in the industry were practicing RL and reasoning models, but DeepSeek pointed out the roadmap to make the industry believe that doing so can really improve intelligence. Improve confidence and attract more AI researchers. Research on turning to new paradigms is of great help.

Only when talents enter, there will be algorithm innovation, and if there is open source pursuit, there will be more investment in computing resources. After DeepSeek, OpenAI, which originally planned to no longer issue new models, issued o3mini one after another, and planned to continue releasing o3, and also considered open source models. Anthropic and Google will also speed up RL research. The industry's advancement of the new paradigm is due to the acceleration of DeepSeek, small and medium-sized teams can also try RL on different domains.

In addition, the improvement of reasoning model will further help the agent to be implemented, and AI researchers are now more confident in the research and exploration of agents. Therefore, it can also be said that DeepSeek open source reasoning model has promoted the industry's further exploration of Agents. .

So DeepSeek has not invented a new paradigm, but has pushed the entire industry into a new paradigm.

Insight03What is the difference between Anthropic's technical route and R1?

From Dario's interview, it can be seen that Anthropic's understanding of R-1/reasoning model is somewhat different from the O series. Dario thinks that base model and reasoning model should be a continuous spectrum, rather than an independent model series such as OpenAI. If you only do the O series, you will soon encounter the ceiling.

I have always wondered why the coding, reasoning and agent capabilities of Sonnet 3.5 have been so strongly improved, but 4o has not caught up with it?

They did a lot of RL work in the pre training base model stage, and the core is to increase the base model, otherwise it may be easy to eat up all the profits by relying solely on RL improvement.

Insight04DeepSeek's sensation is inevitable, and there are also accidents

The Why is Great by two early OpenAI researchers"Can't be planned" describes DeepSeek as well.

From the technical perspective, DeepSeek has the following highlights:

•Open source: Open source is very important. After OpenAI began to transform from GPT-3 to a closed-source company, the first echelon The three giants no longer disclose technical details and gave up a blank open source niche, but Meta and Mistral did not take this position firmly. DeepSeek is a surprise attack this time, and it is smooth in the open source track.

If you give 100 points to sensation, your intelligence enhancement will contribute 30 points to improve intelligence and 70 points to open source. LLaMA was also open source before, but it did not have such a sensational effect, which means that LLaMa's intelligence level is not enough.

•Cheap: The value of the sentence "Your margin is my opportunity" is still rising.

•Internet + Public CoT: These two points of the user can bring a good user experience. DeepSeek's two cards are used at the same time, which can be said to be a king and given to the C-end user. The experience is completely different from other Chatbots. In particular, CoT is transparent, which makes the model thinking process public. Transparency can make users trust AI more and promote breaking the circle. However, in theory, Perplexity has also had a great impact, but the DeepSeek server is unstable. The Perplexity team quickly responded that R-1 was launched and instead took over the users of DeepSeek R-1.

•RL Generalization: Although RL was first proposed by OpenAI o1, because various operations have always been half-covered and the penetration rate is not high. DeepSeek R-1 has greatly promoted reasoning The process of model paradigm has greatly improved its ecological acceptance.

DeepSeek's investment in technology exploration is a deterministic factor that deserves more attention and discussion by this intelligent achievement, but the timing launched by DeepSeek R1 made this sensation occasional:

•In the past, the United States has always said that it is a big lead in basic technology research, but DeepSeek is native to it, which is also a highlight. In this process, many American technology tycoons began to promote DeepSeek to challenge the American technology overlord. The argument of status, DeepSeek is passively involved in the public opinion war;

• Before the release of DeepSeek R1, the OpenAI Stargate $500B incident had just begun to ferment. This huge investment is very contrasted with the intelligent output efficiency of the DeepSeek team. , it is difficult not to attract attention and discussion;

•DeepSeek makes Nvidia's stock priceThe sharp drop further fermented public opinion. They must have never expected that they would become the first black swan of the US stock market at the beginning of 2025;

•The Spring Festival is a training ground for products, and many super apps in the mobile Internet era are during the Spring Festival. The AI ​​era is no exception. DeepSeek R1 happened to be released before the Spring Festival. What surprised the public was its writing ability, rather than the coding and mathematical ability emphasized during training. Cultural and creative products are more likely to be felt by the public and go viral.

Insight05Who is injured? Who benefits?

The players on this arena can be divided into 3 categories: ToC, To Developer and To Enterprise (to Government):

1. ToC: Chatbot is definitely the most impacted, and his mind and brand attention are DeepSeek Snatched it away, ChatGPT is no exception;

2. The influence in the developer link is very limited. We saw that some users said that comments r1 is not easy to use, which is not as good as sonnet. Cursor official also said that Sonnet is still Outperform, users are highly selected for Sonnet, and there is no major migration;

3. In the third dimension, To Enterprise and To Government's business depends on trust and understanding of needs, and large organizations consider the interests of decision-making. Complex, not as easy as C-end users.

Another angle, think about this problem from the perspective of closed source, open source and computing power:

In the short term, everyone will feel that closed source OpenAI/Anthropic/Google is more impacted: p>

•The mysterious technology has been opened, and the premium of the most important mystery in AI Hype has been broken;

•For more realistic factors, the market believes that the potential customers of these closed-source companies are believed to be the most important customers of these closed-source companies. Some of the market sizes have been taken away, and the return cycle of GPU investment has become longer;

•OpenAI, as the leader, is the one who is the most "loss" among them. I wanted to keep the technology unavailable, half of the time I wanted to keep it open. The dream of hoping to earn more technology premium cannot be realized.

But in the medium and long term, companies with abundant GPU resources will still benefit. On the one hand, Meta in the second echelon can quickly follow up on new methods, Capex is more efficient, Meta may be a big beneficiary, and the other In terms of intelligence improvement, more exploration is needed. DeepSeek open source brings everyone's level together, and entering a new exploration requires 10 times or even a large amount of GPU investment.

From the first principle, for the AI ​​intelligent industryWhether it is developing intelligence or applying intelligence, it will inevitably consume massive computing power in terms of physics. This is determined by basic laws and cannot be completely avoided by technical optimization.

So, whether it is exploring intelligence or applying intelligence, even if there are short-term doubts, the demand for computing power in the medium and long term will explode, which also explains why Musk insists on expanding xAI based on the first principles The deep logic behind clustering, xAI and Stargate may be the same. Cloud vendors such as Amazon have announced that they will add Capex guidance.

We assume that the global AI research talent level and cognition are all the same, and more GPUs can be used to do more experimental exploration? In the end, it may still be back to the competition of compute.

DeepSeek Barefoot is not afraid of wearing shoes, has no commercial demands, and focuses on the exploration of AGI intelligent technology. The action of open source is of great significance to promoting the AGI process, intensifying competition, promoting openness, and has a catfish effect. .

Insight06Can distillation surpass SOTA?

There is a very uncertain detail. If DeepSeek has used a lot of distilled CoT data from the pre train stage, then the effect is not amazing today. It is still the basic intelligence obtained on the shoulders of the first-tier giants, and then open source But if there is no large amount of distillation data in the pre train stage, DeepSeek will do pre train from 0 to achieve today's results, that would be amazing.

In addition, it should be unlikely whether distillation can surpass SOTA in base model. But DeepSeek R-1 is very strong, and it is guessed that Reward model does a very good job. If the R-1 Zero path is reliable, there is a chance to surpass SOTA. ,

Insight07No Moat !

Google's previous evaluation of OpenAI: No Moat! This sentence is also very appropriate here.

DeepSeek This wave of Chatbot users has experienced a large number of migrations, which has given the market a very important inspiration: the progress of intelligent technology is very steep, and it is difficult for phased products to form an absolute barrier. .

Whether it is ChatGPT/Sonnet/Perplexity that has just formed a mind and reputation, or developer tools such as Cursor and Windsurf, once a smarter product is available, users will have no idea about the "previous generation" smart products. Loyalty, it is difficult to build a moat today, both in the model layer and the application layer.

DeepSeek also verified one thing this time: the model is the application, DeepSeek has no innovation in product form, the core is intelligence + open source. I can’t help but think: In the AI ​​era, is any innovation in any product or business model not as good as intelligent innovation?

Insight08DepSeek should take over this wave of Chatbot traffic and increase it?

From Chatbot's popularity to today, it can be clearly felt through the reaction of the DeepSeek team that DeepSeek has not yet figured out how to use this wave of traffic.

The essence of the question of whether to catch and actively operate this batch of traffic is that can great business companies and great research labs co-exist in one organization?

This matter is a very test of energy and resource allocation, organizational ability and strategic choice. If it is a large company like Byte and Meta, their first reaction should be followed, and they also have a certain organizational foundation. Next, DeepSeek, as a research lab organization, will inevitably be under great pressure to undertake this huge flow.

But at the same time, I also want to think that this wave of Chatbot will be a phased traffic? Is Chatbot under the main line of future intelligent exploration? It seems that every intelligent stage has a corresponding product form, and Chatbot is just one of the early forms that are unlocked.

For DeepSeek, from the perspective of the next 3-5 years, would it be a miss if we do not undertake Chatbot traffic today? What if there is an economies of scale one day? If AGI is finally implemented, what carrier will it take over?

Insight09Where does the next intelligent breakthrough Aha moment come from?

On the one hand, the next generation model of the first echelon is very critical, but today we are on the limit of Transformer, and it is still uncertain whether the first echelon can come up with a model of intergenerational improvement. OpenAI, Anthropic and Google responded with good 30-50% of the models that may not be enough to save the situation because they have 10-30 times more resources.

On the other hand, the implementation of Agent is more critical, because Agent needs to do long-distance multi-step reasoning. If the model is 5-10%, the leading effect will be magnified many times, so OpenAI, Anthropic and Google On the one hand, we need to implement Agent products, full stack integrated model + Agent products, just like Windows + Office, and on the other hand, we need to show more powerful models, such as the next generation model represented by O3 full version and Sonnet 4/3.5 opus.

The technology is uncertainUnder sex, the most valuable are genius AI researchers. Any organization that wants to explore AGI must invest more radical resources bet on next paradigm, especially in the context of today's pre training stage, which has been pulled by good talents. +Abundant resources and explore the next intelligent emergence of Aha moment.

Insight 10DeepSeek This wave has made me more confident in AI talents and is very encouraging

Finally, I hope that technology has no borders.