DeepSeek open source and releases 3FS, high-speed parallel file system optimizes AI data access
Editor
6 hours ago 6,996
Share to:
According to DeepSeek's announcement, on the fifth day of the open source week, its Fire-Flyer file system (3FS) was officially open source. As a high-performance parallel file system, 3FS can make full use of modern SSD and RDMA networks to achieve high-speed data access and improve AI model training and inference efficiency.
3FS key performance indicators:
Implement 6.6 TiB/s total read throughput in a 180-node cluster;
Achieved 3.66 TiB/min throughput in 25-node GraySort benchmark;
The peak throughput of single-node KVCache query exceeds 40+ GiB/s.
3FS adopts a separate architecture, supports data preprocessing, data set loading, checkpoint storage/recovery, embedded vector search and inference KVCache query, and has strong consistency semantics. DeepSeek simultaneously launched the Smallpond data processing framework to further optimize 3FS data management capabilities.