Toward more economical large-scale foundation models: No longer a game for the few

In recent years, the development of large-scale foundation models (LFMs) has made great advances. However, the high training costs and computational demands have long been a bottleneck for the widespread adoption of this technology. With technological advancements, this situation is undergoing a fundamental transformation. The recent release of DeepSeek-V31 has sparked extensive discussions. Through innovative architectural design and efficient training strategies, it has significantly reduced training costs while achieving performance comparable to top-tier closed-source models. The pre-training cost of DeepSeek-V3 is only $5.576 million, far lower than the hundreds of millions of dollars required for models like GPT-4. As shwon in Figure 1, this breakthrough not only marks the democratization of LFM technology but also opens up opportunities for more small- and medium-sized enterprises and research institutions to participate in AI innovation. In the future, LFMs will no longer be a game for the few.