News center > News > Headlines > Context
a16z "disciple" Kuzco Practical Guide 2: From individual combat to group deployment
Editor
2024-11-28 21:02:01 9,058

a16z

Author: J1N, Techub News

Introduction: Epoch One to Two

Kuzco is a company specializing in serving LLM big languages The model computing power mining network was selected into a16z’s Crypto Startup Accelerator (CSX) autumn accelerator program launched in New York on September 9 this year. Projects selected by the program will receive at least 50 from a16z Ten thousand US dollars of investment, and will receive guidance and support from the a16z operations team. The accelerator program has now ended.

On November 16, Kuzco announced that the first phase (Epoch One) incentive plan will end on November 18, 2024, and all operations will be suspended. The snapshot will be stored permanently, and the final points ranking will be published on the new leaderboard.

Officially disclosed, Epoch One will be launched on March 6, 2024, with a peak number of devices exceeding 8,000. The network runs the 8B specification Llama- released by Meta. 3 AI large language models, with a total reasoning of more than 1 trillion tokens.

And announced that financing information and project development roadmap will be released in the next few weeks, as well as the second phase (Epoch Two) incentive program will be released on December 9 Launching on the same day, Epoch Two will bring some new features, such as higher throughput and reliability of NVIDIA hardware; encourage users to access top computing devices such as A100 and H100; support more image generation and multi-modal language models VLM.

There is still half a month to prepare before the launch of Epoch Two. This article will discuss:

Share the practice and results of personal mining, and the transformation from stand-alone to cluster.

Shows the entire process of obtaining financing through research and practice, and building high-standard machines.

Discuss the matching of hardware configuration and project requirements, and answer common questions from investors.

Epoch One Review: Individual Combat Configuration

The author's configuration list includes RTX series graphics cards 2060, 2070S, 3080, 4060, 4060Ti, as well as 4 4070S and 2 Apple M2 and M3 devices. These devices are distributed on several hosts, laptops and a dedicated mining machine .

Cost

It is worth mentioning that these graphics cards were originally purchased by the author every year for gaming needs, and were not purchased specifically for mining. Therefore, the hardware purchase expenses were not included in the cost calculation, and only the actual cost of the mining machines was calculated. Electricity cost. Here is the first article "a16z "Disciple" Kuzco Practical Guide: How to Efficiently Conduct AI Computing Power Mining? 》Examples of assembled mining machines.

The mining machine configuration:

Mainboard: z490 (industrial board will be replaced later)

p>

CPU: 10th generation I9

Graphics cards: 2060, 2070s, 3080, 4060ti, 4070s

Hand-rubbed mining machine

The picture below shows the power consumption of the mining machine in October and November. A total of 564 kilowatt hours was obtained. Points were obtained (KZO Point) is approximately 600 million points. All machines combined add up to about 1.1 billion points. The specific electricity cost needs to be calculated based on the electricity bill in your location. This is only for reference.

< em>On the far right of the picture, a total of 1 billion points have been obtained

Preparing for Epoch Two: Cluster Deployment

Based on what the author shared in the first article and his rich operation and maintenance experience in personally participating in equipment assembly, debugging and environment deployment, the author successfully obtained a certain amount of financial support and invested it all in assembling high-performance mining equipment. machine to further enhance computing power scale and operational efficiency.

Single-hand deployment to cluster deployment

Configuration and selection logic of high-specification machines

Combined with the author's experience in Epoch One Based on practical experience, we have comprehensively optimized the motherboard, CPU, graphics card, power supply, platform and network configuration, and selected a more suitable hardware combination. This not only improves the stability, security and efficiency of the overall operation, but also improves the hardware selection. This strategy can effectively reduce the actual investment cost and provide higher cost-effective options for subsequent participants.

Motherboard

The author chose an industrial motherboard instead of the mainstream B85, mainly based on comprehensive considerations of performance, stability and cost-effectiveness.

In terms of performance, Llama running Kuzco The -3 model needs to start multiple Docker processes, and running these processes in parallel will occupy a lot of CPU resources and require high CPU performance, and B85-compatible CPUs cannot meet this demand.

In addition, industrial motherboards have obvious advantages in long-term stable operation, high temperature resistance and manufacturer warranty. At the same time, they have stronger liquidity in the second-hand market, so they are undoubtedly the best choice.

Graphics cards

The author chose to use 4070S as the main graphics card, mainly based on the following points:

Advantages of AI computing performance: compared For 30 series graphics cards, 40 The performance improvement of series graphics cards in AI computing is much greater than the improvement in game performance. The core reason is that the AI ​​computing power mainly depends on the number of CUDA cores of the graphics card, and the CUDA cores of the 40 series graphics cards are significantly more than that of the 30 series graphics cards.

Energy efficiency advantage: The author conducted detailed tests on a variety of GPUs and calculated the average power consumption of each Tokens

4060Ti (160W): 0.125 Tokens/W

3080 (330W): 0.22 Tokens/W

4090 (450W): 0.26 Tokens/W

4070S (220W): 0.38 Tokens/W

From the test results See, 4070S has the best balance of performance and power consumption, and its higher energy efficiency ratio directly reduces electricity costs, making it the most cost-effective option.

Price and liquidity in the second-hand market: As a mid-to-high-end graphics card, 4070S has high liquidity and value preservation in the second-hand market, further reducing the maintenance cost of the equipment. cost effective while providing flexibility for subsequent hardware upgrades.

CPU

As mentioned earlier, Kuzco's Llama-3 needs to start multiple Dockers when running, which takes up a significant amount of CPU resources, especially when running multiple Docker servers. When the card is running, the CPU usage may be as high as 80%-90%. Therefore, multi-core and multi-thread processing capabilities are particularly important. The high-performance, multi-threaded, and stable CPU can not only effectively support multi-task operations, but also ensure the stability and efficiency of the entire mining process.

< em>The 13th generation i5 can reach 70%+ occupancy when running the graphics card at full load

Network environment

Soft routing is the square box in the picture

The network environment is also crucial in mining, even if it is configured with high Performance graphics card, if the network is not optimized, the computing power will also be seriously affected. According to the author's actual measurement, insufficient network speed may cause the computing power to drop to 30%, and low-quality network nodes may directly lead to the inability to connect to the Kuzco network. Both of these points are unacceptable for mining. In order to solve these problems, the author adopts a soft routing solution. This method is not only easy to configure, but also can run efficiently without manual intervention after completing the setup. In theory, it can also support the access of unlimited devices. As for the specific operation methods, readers are recommended to consult relevant information according to their needs.

Power supply

Classic Great Wall 2000w Nuclear Bomb Power Supply

When choosing a power supply, you need to pay special attention to the issue of peak power consumption. This is why even though the rated power consumption of the 7-photo 4070S is only 1540W, the author still chooses to use dual 2000W power supplies, with a total power of 4000W. This is not a waste of resources, but is out of consideration for the stability and safety of the equipment.

The graphics card will experience peak power consumption during operation, that is, its actual power consumption may reach 1.5 times or even more than the rated power consumption at some moments, and then it will back to normal levels. If the power supply is not powerful enough to handle this peak, it may trigger a forced shutdown mechanism of the power supply and even cause damage to the graphics card. This is a fatal threat to the normal operation of the mining machine.

4070s running power consumption performance

< p style="text-align: left;">Take 4070S as an example. Although its rated power consumption is 220W, the peak power consumption may exceed 400W. The total peak power consumption of 7 graphics cards may reach more than 3000W, so dual 2000W power supplies are configured to ensure the stable operation of the machine. Users configuring multiple 4090s need to be especially aware that the rated power consumption of a single 4090 is 450W, and the peak power consumption may be as high as 770W. In the case of multiple cards, only two power supplies may not be able to meet the demand. In this case, three power supplies are usually required to ensure system stability.

4090 running power consumption performance

< p style="text-align: left;">Supplement

As for BIOS settings, hardware compatibility, remote management and other issues, the author will not go into too much detail here. . There are a large number of free tutorials on the Internet for reference. Most problems can be solved by following the tutorials. It is recommended to conduct targeted review and processing according to your own hardware configuration and needs, which is simple and efficient.

Risk and Return

Answer the question that everyone is most concerned about: How much money can be mined every day? Frankly speaking,There is no clear answer to this question because risks and benefits always coexist. I can share a clear point of view: whether it is the currency circle or the traditional industry, if any project can accurately calculate the daily income, then you will probably not be able to make a lot of money. Unless you have certain monopoly resources, such as extremely low electricity costs or very cheap mining equipment, you can have an advantage in revenue. However, not everyone has such resources.

The author chooses equipment with good liquidity precisely to reduce investment risks and cost pressure. Taking Kuzco mining as an example, the cost is mainly concentrated on the depreciation of the hardware and electricity costs, so your maximum loss is limited to these fixed costs. If participation is not carried out at a low cost, then any investment decision loses its meaning. It needs to be emphasized that the characteristics of head mining determine that there is no clear profit expectation, but this is also where the potential of head mining lies.

From a subjective judgment, this track has huge market prospects: on the one hand, Kuzco has received investment support from a16z; on the other hand, LLM large language The demand for models is expanding rapidly. Think about it, almost no one doesn’t use LLM, right? Rounds of high-dollar funding rounds for platforms like OpenAI’s ChatGPT, Meta’s Llama, and Musk’s XAI clearly demonstrate the growth potential of this industry.

For ordinary people, it is not easy to directly participate in the AI ​​industry. On the one hand, the threshold for AI technology is high; on the other hand, the training of AI models requires a huge amount of resources and funds, and most people cannot afford such costs. By joining the AI ​​computing power network through Kuzco, ordinary people can easily participate in this high-growth field at a controllable cost, contribute to AI computing power, and earn profits at the same time.

In addition, the price of Bitcoin is currently about to exceed US$100,000, rising from US$16,000 in 2022 to today’s highs, and there is a huge retracement behind it. risk. If you choose to purchase tokens of AI projects directly, you will also face similar high volatility risks. In contrast, participating in the AI ​​computing power network is a more robust option: not only is the cost clearly controllable, but it can also enter the high-speed growth track of the AI ​​industry with relatively low risk. This is one of the practical ways for ordinary people to enter the field of AI in the current environment.

Keywords: Bitcoin
Share to: