The affordability of DeepSeek is a myth: The revolutionary AI actually cost $1.6 billion to develop

DeepSeek's new chatbot boasts an impressive introduction: "Hi, I was created so you can ask anything and get an answer that might even surprise you." This AI, a product of the Chinese startup DeepSeek, has rapidly become a major player, even causing significant drops in NVIDIA's stock price.

DeepSeek's competitive edge lies in its innovative architecture and training methods. Key technologies include:

• Multi-token Prediction (MTP): Instead of predicting words individually, MTP forecasts multiple words simultaneously, boosting accuracy and efficiency.
• Mixture of Experts (MoE): This architecture uses multiple neural networks (256 in DeepSeek V3, with eight activated per token), accelerating training and enhancing performance.
• Multi-head Latent Attention (MLA): MLA repeatedly focuses on key sentence parts, minimizing the risk of overlooking crucial information.

DeepSeek's initial claim of a mere $6 million training cost for DeepSeek V3, using only 2048 GPUs, has been challenged. SemiAnalysis revealed a far more extensive infrastructure, encompassing approximately 50,000 Nvidia Hopper GPUs (including 10,000 H800s, 10,000 H100s, and additional H20s) spread across multiple data centers. This translates to a server investment of roughly $1.6 billion and operational expenses estimated at $944 million.

DeepSeek, a subsidiary of High-Flyer, a Chinese hedge fund, owns its data centers, fostering control and innovation. Its self-funded nature allows for rapid decision-making. Furthermore, the company attracts top talent, with some researchers earning over $1.3 million annually, primarily from Chinese universities.

While DeepSeek's $6 million training cost claim is misleading (reflecting only pre-training GPU usage, excluding research, refinement, data processing, and infrastructure), the company has invested over $500 million in AI development. Its lean structure facilitates efficient innovation.

DeepSeek's success demonstrates the potential of a well-funded, independent AI company to compete with industry giants. However, its achievements are built on substantial investment, technical advancements, and a strong team, making the "revolutionary budget" narrative an oversimplification. Even so, DeepSeek's costs remain significantly lower than competitors. For example, DeepSeek's R1 model cost $5 million to train, compared to ChatGPT4's $100 million.