$5 million total is false. It's obvious the people who say it's true know little to nothing about AIDL or software engineering. It's like telling a non runner someone ran a 2 minute 5k. The non runner is oblivious.
A well trained pretrained model (the P in GPT for example) alone costs significantly more than $5 million in R&D. For the deluded people out there, that is just the next word predictor, the model before the model. A multilayered feed forward neural net outputs a probability distribution of the words in its vocabulary based on the previous words. This type of model is not new, has been around for decades, and easily surpasses that fictitious $5 million number.
That Ben Thompson DeepSeek FAQ (post #43 above) addresses that:
Ben Thompson wrote:
I’m not sure I understood any of that.
The key implications of these breakthroughs — and the part you need to understand — only became apparent with V3, which added a new approach to load balancing (further reducing communications overhead) and multi-token prediction in training (further densifying each training step, again reducing overhead): V3 was shockingly cheap to train. DeepSeek claimed the model training took 2,788 thousand H800 GPU hours, which, at a cost of $2/GPU hour, comes out to a mere $5.576 million.
That seems impossibly low.
DeepSeek is clear that these costs are only for the final training run, and exclude all other expenses; from the V3 paper:
Lastly, we emphasize again the economical training costs of DeepSeek-V3, summarized in Table 1, achieved through our optimized co-design of algorithms, frameworks, and hardware. During the pre-training stage, training DeepSeek-V3 on each trillion tokens requires only 180K H800 GPU hours, i.e., 3.7 days on our cluster with 2048 H800 GPUs. Consequently, our pre- training stage is completed in less than two months and costs 2664K GPU hours. Combined with 119K GPU hours for the context length extension and 5K GPU hours for post-training, DeepSeek-V3 costs only 2.788M GPU hours for its full training. Assuming the rental price of the H800 GPU is $2 per GPU hour, our total training costs amount to only $5.576M. Note that the aforementioned costs include only the official training of DeepSeek-V3, excluding the costs associated with prior research and ablation experiments on architectures, algorithms, or data.
So no, you can’t replicate DeepSeek the company for $5.576 million.
I still don’t believe that number.
Actually, the burden of proof is on the doubters, at least once you understand the V3 architecture. Remember that bit about DeepSeekMoE: V3 has 671 billion parameters, but only 37 billion parameters in the active expert are computed per token; this equates to 333.3 billion FLOPs of compute per token. Here I should mention another DeepSeek innovation: while parameters were stored with BF16 or FP32 precision, they were reduced to FP8 precision for calculations; 2048 H800 GPUs have a capacity of 3.97 exoflops, i.e. 3.97 billion billion FLOPS. The training set, meanwhile, consisted of 14.8 trillion tokens; once you do all of the math it becomes apparent that 2.8 million H800 hours is sufficient for training V3. Again, this was just the final run, not the total cost, but it’s a plausible number.
Yes, that is how I interpreted the number and it was very clear from published documents what the $5M meant.
It’s not shocking in retrospect. Deepseek is trading off inference time for training time compute. That plus mixture models and other lower-level reported optimizations are all consistent with their superior cost scaling. These optimizations are not exactly earth-shattering or unknown to experts in retrospect, but Americans were horseblindered by not having the constraints the Chinese did. What the Chinese accomplished is technically quite cool, yet there is nothing to panic about for US tech companies or even Nvidia, market whimsicalities notwithstanding.
1) Isn't this a sign that Weldon has already been in San Fran too long? He's freaking out that pre-IPO silicon valley firm just took a 50% haircut. Meanwhile, we won't have to ruin the worldwide water supply to make AI moving forward.
2) I just hope this means we are one-step closer to Sam Altman being made irrelevant. Can't stand that guy.
3) If you are into technology questions like this, you really should pay for stratechery. Even if yo don't pay, you can get one article for free a week including Ben Thompson's most recent Q&A about Deep Seek (that ohters have already cited) here:
I think deepseek is somewhat overhyped. Their models are not quite as good as the best Western ones and the methods they used to achieve better efficiency can definitely be copied
I partially agree with this but there are some notable exceptions. Bell Labs was the R&D arm of a telephone monopoly which invented all sorts of incredible things in large part because of their monopolistic position.
I think Google and Facebook had the opportunity to do something similar but their "inventions" were DEI and ESG projects and thus will have near-zero long term technological impact.
If you actually think about Google, they are basically still a search engine that got better at advertising. And if you think about Facebook, they're still a social network that also just got better at advertising (I personally think their primary product, a social network, has gotten significantly worse over the past two decades).
The opposite of Google/Facebook would be someone like Elon Musk who astonishingly at every turn finds a way to push something new.
We need monopolistic powers for innovators like Musk and anti-trust action against low-innovation companies like Google/Facebook.
The Lina Khan's of the world, with their hard-left, DEI/ESG progressive beliefs, should not be in positions of power to be making these decisions.
The point is not that monopolistic firms never produce innovation. In fact, Chinese firms are granted their own kinds of monopolistic protection in order to pursue innovation (typically, for the period of the renewable 5 year plan). It's that the investment model that produced today's industry-dominating US tech firms is not as efficient at producing innovation in the long run-- and, in particular, innovation that directly addresses pressing, systemic needs (which they won't or can't address, and which they refuse to pay sufficient tax so that the state can address them).
Then there's the question of the ability of these giant firms to rent-seek against the public interest, thus throttling innovation and hogging disposable income that could be spent on other, growth-producing things (something which Lina Khan was attempting to directly address-- the whole panic over "wokeness" is/was a deflection. These firms were already pursuing these kinds of policies on their own before it became politically unfashionable, and no one can actually show that it had an effect on their capacity to innovate).
The US model of large scale industrial development since at least the 1990s is essentially based on producing massive investment bubbles (many, many start-up firms competing for funding, mostly from a few huge financial conglomerates) with the hope that, when the bubble bursts, a few very profitable winners will be left standing, and will go on to dominate their industries, and thus be free to plan-out further innovation over decades. See, e.g., the dot.com and finance industry bubbles from the mid-90s to the crash in '08.
The trouble has been that as soon as these firms have achieved their dominant positions, they have put rent-seeking and rewarding shareholders ahead of investment in innovation, and have participated in pumping up subsequent investment bubbles around new products, some of them of unproven long term value (think of Meta's virtual reality promise, the whole crypto industry, Musk's self-driving cars and trucks, and, yes, AI). China's ability to abruptly and decisively cut the legs out from under an emerging and massively capitalized US industry is a stark illumination of the flaws of the US investment model and of the kinds of firms it has produced.
Add to this the obscene amounts of wealth the US burns on "defense spending"-- which used to produce tech innovation that was then given over gratis to private companies, but is now simply a form of corporate welfarism for a few massive firms-- and it's not hard to see how China could be in the process of overtaking the US in meaningful innovation.
The Chinese have been forced by US predominance and protectionism to do a form of very large scale industrial policy, directly targeting the kinds of innovation that will actually address current and future global problems. This just might be a better way of doing capitalism than leaving innovation to the whims of giant monopoly firms and highly skewed financial markets. And there's no question that the wider availability (in both the US and the world) of cheaper and equal-quality Chinese EVs and green energy tech would go a long way to solving the one massive, existential problem that free market capitalism simply cannot solve-- climate change.
In short, the US is in a crisis of capitalist leadership. Unlike the leading capitalists of the first US Gilded Age, today's Robber Barons only rob. They have zero concern for or vision of the common good (in part why they were ultimately fine with Trump). Your Zuckerburgs and your Gates set up so-called charitable funds that really exist to avoid taxation, and the rest don't even do that much! Imagine even one of these people doing the equivalent of building 1,700 lavish public libraries across the country, which is what Robert Carnegie did of his own accord between 1880 and 1929. And this is just one example.
I’ve seen multiple back of the envelope calculations that the cost of the training run could have been $5M. They are clear this is for the model alone. Obviously DeepSeek has a bigger budget. Their paper on arxiv has 200 authors!
The big advances seem to be in multi-token prediction, reinforcement learning, and complicated stuff to improve efficiency. Writing this off as “copying”, as I’ve said, is wrong. This model clearly has some major advancements that the US labs have not released yet.
The debate about the true cost is kind of pointless IMO. This is a state of the art model and shows that China is not behind in AI and is making fundamental advancements. This is absolutely an arms race now. NVDA selling off is kind of funny in that light.
This post was edited 25 seconds after it was posted.
I partially agree with this but there are some notable exceptions. Bell Labs was the R&D arm of a telephone monopoly which invented all sorts of incredible things in large part because of their monopolistic position.
I think Google and Facebook had the opportunity to do something similar but their "inventions" were DEI and ESG projects and thus will have near-zero long term technological impact.
If you actually think about Google, they are basically still a search engine that got better at advertising. And if you think about Facebook, they're still a social network that also just got better at advertising (I personally think their primary product, a social network, has gotten significantly worse over the past two decades).
The opposite of Google/Facebook would be someone like Elon Musk who astonishingly at every turn finds a way to push something new.
We need monopolistic powers for innovators like Musk and anti-trust action against low-innovation companies like Google/Facebook.
The Lina Khan's of the world, with their hard-left, DEI/ESG progressive beliefs, should not be in positions of power to be making these decisions.
The point is not that monopolistic firms never produce innovation. In fact, Chinese firms are granted their own kinds of monopolistic protection in order to pursue innovation (typically, for the period of the renewable 5 year plan). It's that the investment model that produced today's industry-dominating US tech firms is not as efficient at producing innovation in the long run-- and, in particular, innovation that directly addresses pressing, systemic needs (which they won't or can't address, and which they refuse to pay sufficient tax so that the state can address them).
Then there's the question of the ability of these giant firms to rent-seek against the public interest, thus throttling innovation and hogging disposable income that could be spent on other, growth-producing things (something which Lina Khan was attempting to directly address-- the whole panic over "wokeness" is/was a deflection. These firms were already pursuing these kinds of policies on their own before it became politically unfashionable, and no one can actually show that it had an effect on their capacity to innovate).
The US model of large scale industrial development since at least the 1990s is essentially based on producing massive investment bubbles (many, many start-up firms competing for funding, mostly from a few huge financial conglomerates) with the hope that, when the bubble bursts, a few very profitable winners will be left standing, and will go on to dominate their industries, and thus be free to plan-out further innovation over decades. See, e.g., the dot.com and finance industry bubbles from the mid-90s to the crash in '08.
The trouble has been that as soon as these firms have achieved their dominant positions, they have put rent-seeking and rewarding shareholders ahead of investment in innovation, and have participated in pumping up subsequent investment bubbles around new products, some of them of unproven long term value (think of Meta's virtual reality promise, the whole crypto industry, Musk's self-driving cars and trucks, and, yes, AI). China's ability to abruptly and decisively cut the legs out from under an emerging and massively capitalized US industry is a stark illumination of the flaws of the US investment model and of the kinds of firms it has produced.
Add to this the obscene amounts of wealth the US burns on "defense spending"-- which used to produce tech innovation that was then given over gratis to private companies, but is now simply a form of corporate welfarism for a few massive firms-- and it's not hard to see how China could be in the process of overtaking the US in meaningful innovation.
The Chinese have been forced by US predominance and protectionism to do a form of very large scale industrial policy, directly targeting the kinds of innovation that will actually address current and future global problems. This just might be a better way of doing capitalism than leaving innovation to the whims of giant monopoly firms and highly skewed financial markets. And there's no question that the wider availability (in both the US and the world) of cheaper and equal-quality Chinese EVs and green energy tech would go a long way to solving the one massive, existential problem that free market capitalism simply cannot solve-- climate change.
In short, the US is in a crisis of capitalist leadership. Unlike the leading capitalists of the first US Gilded Age, today's Robber Barons only rob. They have zero concern for or vision of the common good (in part why they were ultimately fine with Trump). Your Zuckerburgs and your Gates set up so-called charitable funds that really exist to avoid taxation, and the rest don't even do that much! Imagine even one of these people doing the equivalent of building 1,700 lavish public libraries across the country, which is what Robert Carnegie did of his own accord between 1880 and 1929. And this is just one example.
Very well said on multiple fronts. I'll add that our capitalistic system is driven by "shareholder value syndrome" in most all sectors. We seem to have lost the ability innovate, do good for society, provide decent jobs for the masses, a comfortable (not obscene) pay for the plutocrats at the same time and doesn't appear we have any desire to do so. You mention climate change...for us it's wealth over health.
Seeing some misunderstanding about basic terms like "open source" in this thread (which is critical to understand why event is so significant). But also some very knowledgable takes, which is nice to see.
Here's what you need to know:
1) In terms of performance on standard benchmarks, DeepSeek is about as good as our best-of-the-best models
2) In terms of efficiency, DeepSeek is far more computationally efficient to use. This is not up for debate. It can be easily measured.
3) In terms of training efficiency, the jury is still out on exactly how much more efficiently this model was trained, but it appears that the $6 million figure is technically true, but with some major caveats.
4) China just gave the entire world free and complete access to a technology that the U.S. and U.S. companies have been trying to control and monetize. Makes the U.S. look greedy, silly, and backwards.
Lastly, a word on censorship. The default DeepSeek model is censored to filter out criticisms of the Chinese government. But that's the beauty of open source - with some very minor fine tuning, you can easily bypass these filters.
In short, China just dunked on the U.S. economically, politically, and psychologoically. Not a standard in-game dunk. Michael Jordan 1993 NBA Jam jumping 50 feet in the air while doing a 360 dunk.
I partially agree with this but there are some notable exceptions. Bell Labs was the R&D arm of a telephone monopoly which invented all sorts of incredible things in large part because of their monopolistic position.
I think Google and Facebook had the opportunity to do something similar but their "inventions" were DEI and ESG projects and thus will have near-zero long term technological impact.
If you actually think about Google, they are basically still a search engine that got better at advertising. And if you think about Facebook, they're still a social network that also just got better at advertising (I personally think their primary product, a social network, has gotten significantly worse over the past two decades).
The opposite of Google/Facebook would be someone like Elon Musk who astonishingly at every turn finds a way to push something new.
We need monopolistic powers for innovators like Musk and anti-trust action against low-innovation companies like Google/Facebook.
The Lina Khan's of the world, with their hard-left, DEI/ESG progressive beliefs, should not be in positions of power to be making these decisions.
The point is not that monopolistic firms never produce innovation. In fact, Chinese firms are granted their own kinds of monopolistic protection in order to pursue innovation (typically, for the period of the renewable 5 year plan). It's that the investment model that produced today's industry-dominating US tech firms is not as efficient at producing innovation in the long run-- and, in particular, innovation that directly addresses pressing, systemic needs (which they won't or can't address, and which they refuse to pay sufficient tax so that the state can address them).
Then there's the question of the ability of these giant firms to rent-seek against the public interest, thus throttling innovation and hogging disposable income that could be spent on other, growth-producing things (something which Lina Khan was attempting to directly address-- the whole panic over "wokeness" is/was a deflection. These firms were already pursuing these kinds of policies on their own before it became politically unfashionable, and no one can actually show that it had an effect on their capacity to innovate).
The US model of large scale industrial development since at least the 1990s is essentially based on producing massive investment bubbles (many, many start-up firms competing for funding, mostly from a few huge financial conglomerates) with the hope that, when the bubble bursts, a few very profitable winners will be left standing, and will go on to dominate their industries, and thus be free to plan-out further innovation over decades. See, e.g., the dot.com and finance industry bubbles from the mid-90s to the crash in '08.
The trouble has been that as soon as these firms have achieved their dominant positions, they have put rent-seeking and rewarding shareholders ahead of investment in innovation, and have participated in pumping up subsequent investment bubbles around new products, some of them of unproven long term value (think of Meta's virtual reality promise, the whole crypto industry, Musk's self-driving cars and trucks, and, yes, AI). China's ability to abruptly and decisively cut the legs out from under an emerging and massively capitalized US industry is a stark illumination of the flaws of the US investment model and of the kinds of firms it has produced.
Add to this the obscene amounts of wealth the US burns on "defense spending"-- which used to produce tech innovation that was then given over gratis to private companies, but is now simply a form of corporate welfarism for a few massive firms-- and it's not hard to see how China could be in the process of overtaking the US in meaningful innovation.
The Chinese have been forced by US predominance and protectionism to do a form of very large scale industrial policy, directly targeting the kinds of innovation that will actually address current and future global problems. This just might be a better way of doing capitalism than leaving innovation to the whims of giant monopoly firms and highly skewed financial markets. And there's no question that the wider availability (in both the US and the world) of cheaper and equal-quality Chinese EVs and green energy tech would go a long way to solving the one massive, existential problem that free market capitalism simply cannot solve-- climate change.
In short, the US is in a crisis of capitalist leadership. Unlike the leading capitalists of the first US Gilded Age, today's Robber Barons only rob. They have zero concern for or vision of the common good (in part why they were ultimately fine with Trump). Your Zuckerburgs and your Gates set up so-called charitable funds that really exist to avoid taxation, and the rest don't even do that much! Imagine even one of these people doing the equivalent of building 1,700 lavish public libraries across the country, which is what Robert Carnegie did of his own accord between 1880 and 1929. And this is just one example.
You are saying a lot of different things colored by your political biases. But there is no problem with the American model of venture capital and the (mostly) free market driving innovation. There is no reason to think that state planning can do better than the profit motive of decentralized actors with wealth in innovation.
The US is still the best in academic and industrial R&D and will continue to draw the best in the world for a long time to come. That brain trust is ultimately the single most important determiner of who gets to be at the frontier of technology innovation.
There is something to be said about big tech having an uncomfortably large amount of power. But becoming big hasn’t slowed down innovation, quite the contrary. Google is a great example of a company that still makes a frighteningly high fraction of its revenue from one thing and yet has probably been the most innovative company for the last couple decades in technology development (but not necessarily commercialization).
I partially agree with this but there are some notable exceptions. Bell Labs was the R&D arm of a telephone monopoly which invented all sorts of incredible things in large part because of their monopolistic position.
I think Google and Facebook had the opportunity to do something similar but their "inventions" were DEI and ESG projects and thus will have near-zero long term technological impact.
If you actually think about Google, they are basically still a search engine that got better at advertising. And if you think about Facebook, they're still a social network that also just got better at advertising (I personally think their primary product, a social network, has gotten significantly worse over the past two decades).
The opposite of Google/Facebook would be someone like Elon Musk who astonishingly at every turn finds a way to push something new.
We need monopolistic powers for innovators like Musk and anti-trust action against low-innovation companies like Google/Facebook.
The Lina Khan's of the world, with their hard-left, DEI/ESG progressive beliefs, should not be in positions of power to be making these decisions.
Google and Meta both have state of the art AI products. How is that not innovation?
Google has rolled out self driving cars in SF and Phoenix (maybe others?) that actually work.
Meta has by far the best VR work.
Google DeepMind built superhuman chess, go, SC2 engines.
Deep Seek is cool and nice. But GPUs spend most of it;s cycles moving data about. We need Electronic Neurons to make things really fast. No commercial chip maker has it. Rumors say Japan Mil has it but won't confirm or answer.
It is literally Chat gpt just tweaked in significant ways for some large improvements. If you ask Deepseek what it is, it will till you something along the lines of "I am Chat GPT"
It is literally Chat gpt just tweaked in significant ways for some large improvements. If you ask Deepseek what it is, it will till you something along the lines of "I am Chat GPT"
And that's precisely the point.
Do you know what the term "open source" means in this context? Can you explain why OpenAI is not open source and DeepSeek is. Can you explain why that is significant?
If you cannot, then you don't understand this story.
I partially agree with this but there are some notable exceptions. Bell Labs was the R&D arm of a telephone monopoly which invented all sorts of incredible things in large part because of their monopolistic position.
I think Google and Facebook had the opportunity to do something similar but their "inventions" were DEI and ESG projects and thus will have near-zero long term technological impact.
If you actually think about Google, they are basically still a search engine that got better at advertising. And if you think about Facebook, they're still a social network that also just got better at advertising (I personally think their primary product, a social network, has gotten significantly worse over the past two decades).
The opposite of Google/Facebook would be someone like Elon Musk who astonishingly at every turn finds a way to push something new.
We need monopolistic powers for innovators like Musk and anti-trust action against low-innovation companies like Google/Facebook.
The Lina Khan's of the world, with their hard-left, DEI/ESG progressive beliefs, should not be in positions of power to be making these decisions.
Google and Meta both have state of the art AI products. How is that not innovation?
Google has rolled out self driving cars in SF and Phoenix (maybe others?) that actually work.
Meta has by far the best VR work.
Google DeepMind built superhuman chess, go, SC2 engines.
DeepMind just won a Nobel prize for AlphaFold.
Low innovation???
Again, it's a matter of comparison and of opportunity costs compared with other conceivable models, including the one that China now seems to represent.
Of course US companies will continue to produce innovation, but they will do so according to the logic of the funding/investment model I described, which has some acute costs that go along with any innovation, not least of which is the extreme potential for rent-seeking against the interests of both consumers and potential competitors in innovation. (And it's a counterfactual, but we need to consider what rate and type of innovation and what social outcomes might have obtained these past 30-odd years under a different system, even one which did the minimum of systematically dismantling massive, anti-competitive firms before they had the ability to buy up and throttle their emerging competition).
And about the whole "Sputnik Moment" thing, there's a temptation to see Chinese AI as representing an a new "arms race". But this analogy is flawed. Rivalry with the old Soviet Union immediately took the form of an actual arms race (which turned into a global inter-imperialist rivalry) because the historical context was one in which both systems had brand new military industrial complexes (and in part because both society's understanding of the consequences of war were naive, being rooted the experience of WWII.)
US-China rivalry need not be this kind of zero-sum game-- that is, if the US responds by reforming its whole regime of national investment in order to successfully compete with Chinese innovation. The US's other alternatives are to: 1. Hope that the Chinese model will not continue to work in the long run; and/or 2. Attempt to deploy its residual economic, geostrategic, and military predominance to forcibly deny China the fruits of what might turn out to be its superior regime of late capitalist innovation.
There is no question that the world would be better off if, in the end, the US was able to tame the power of its monopolies and put the oligarchs who control them in their place in order to better compete with China economically rather than militarily, or by other extra-economic means. Choosing the latter course (around which, at the moment, there seems to be a bipartisan consensus) will lead to both pressure towards military conflict and retarded net economic global productivity and overall well being.
In many ways the fate of settled life as we know it depends largely on the ability of the US ruling class (in both its economic and political manifestations) to reform itself. And so far, the odds of this ever happening don't look great.
This post was edited 1 minute after it was posted.
Google and Meta both have state of the art AI products. How is that not innovation?
Google has rolled out self driving cars in SF and Phoenix (maybe others?) that actually work.
Meta has by far the best VR work.
Google DeepMind built superhuman chess, go, SC2 engines.
DeepMind just won a Nobel prize for AlphaFold.
Low innovation???
Again, it's a matter of comparison and of opportunity costs compared with other conceivable models, including the one that China now seems to represent.
Of course US companies will continue to produce innovation, but they will do so according to the logic of the funding/investment model I described, which has some acute costs that go along with any innovation, not least of which is the extreme potential for rent-seeking against the interests of both consumers and potential competitors in innovation. (And it's a counterfactual, but we need to consider what rate and type of innovation and what social outcomes might have obtained these past 30-odd years under a different system, even one which did the minimum of systematically dismantling massive, anti-competitive firms before they had the ability to buy up and throttle their emerging competition).
And about the whole "Sputnik Moment" thing, there's a temptation to see Chinese AI as representing an a new "arms race". But this analogy is flawed. Rivalry with the old Soviet Union immediately took the form of an actual arms race (which turned into a global inter-imperialist rivalry) because the historical context was one in which both systems had brand new military industrial complexes (and in part because both society's understanding of the consequences of war were naive, being rooted the experience of WWII.)
US-China rivalry need not be this kind of zero-sum game-- that is, if the US responds by reforming its whole regime of national investment in order to successfully compete with Chinese innovation. The US's other alternatives are to: 1. Hope that the Chinese model will not continue to work in the long run; and/or 2. Attempt to deploy its residual economic, geostrategic, and military predominance to forcibly deny China the fruits of what might turn out to be its superior regime of late capitalist innovation.
There is no question that the world would be better off if, in the end, the US was able to tame the power of its monopolies and put the oligarchs who control them in their place in order to better compete with China economically rather than militarily, or by other extra-economic means. Choosing the latter course (around which, at the moment, there seems to be a bipartisan consensus) will lead to both pressure towards military conflict and retarded net economic global productivity and overall well being.
In many ways the fate of settled life as we know it depends largely on the ability of the US ruling class (in both its economic and political manifestations) to reform itself. And so far, the odds of this ever happening don't look great.
my post was more pointing out the dumb DEI complaining the other poster was doing. But still:
I have a hard time understanding the framing of “monopolies” in terms of US tech and AI research. There’s currently intense competition among 5+ US tech firms with many of them deploying $10s of billions of capital per year. Nothing about that is monopolistic. Cutting edge AI research requires huge amounts of capital so it’s naturally going to be done by the biggest companies — but smaller orgs can still find room to innovate and competez
This absolutely is an arms race. The use cases of advanced AI include weapons and war just as the use cases for orbital rockets did in the 50s.
China is quite clear in their goals to break up US hegemony and overtake them. There’s nothing wrong with chip bans and other policies if you think the US staying ahead in the AI race is valuable. It think it is - I think you have to pragmatically assess China as a rival and avoid fairy tales of de-escalation and cooperation.
Lastly, I don’t see the evidence for the “Chinese model” being better than the US research system. Certainly DeepSeek is impressive and close to parity, but the bulk of key advancements have still been made by American research teams. They’re certainly competitive but saying “this is a sign the US needs to rethink its whole economic structure” is hyperbole.
"1) How do we know they actually built for $5 million what others spent billions on?"
We don’t but they released the code as open source and we are already testing it as we speak. We will prove it yes or no in less than a week.
WTF is “2”
"TLDR: 1) Do you trust the claims being made 2) Isn't this a good thing long term?"
The timing was very suspicious after Trump’s announcement. So that was geopolitical manipulation.
Oh, it’s a very good thing and only fools would short Nvidia. Making it cheaper means it will be implemented more, this is Jevon’s paradox.
I’m running it right now on an Intel i9-14900KF, 64 gigs RAM, and an AMD Radeon RX 7900 XTX. I think they are telling the truth. But that does not matter. It’s out there now…..for North Korea, Russia….and whoever…..
"1) How do we know they actually built for $5 million what others spent billions on?"
We don’t but they released the code as open source and we are already testing it as we speak. We will prove it yes or no in less than a week.
WTF is “2”
"TLDR: 1) Do you trust the claims being made 2) Isn't this a good thing long term?"
The timing was very suspicious after Trump’s announcement. So that was geopolitical manipulation.
Oh, it’s a very good thing and only fools would short Nvidia. Making it cheaper means it will be implemented more, this is Jevon’s paradox.
I’m running it right now on an Intel i9-14900KF, 64 gigs RAM, and an AMD Radeon RX 7900 XTX. I think they are telling the truth. But that does not matter. It’s out there now…..for North Korea, Russia….and whoever…..
They’ve been steadily announcing releases since late 2023. There was one last month in December as well that was comparable to o1, just less convincingly so as the recent release. The arxiv paper describing training chips cost details of V3 is dated Dec 27.
There is little reason to think the timing had anything at all to do with Trump, nor is it the first open source LLM available to Russia or North Korea or anyone.
DeepSeek (Chinese: 深度求索; pinyin: Shēndù Qiúsuǒ) is a Chinese artificial intelligence company that develops open-source large language models (LLM). Based in Hangzhou, Zhejiang, it is owned and solely funded by Chinese hed...
Meta VR hallucinates so much at the $10 Billion NFL Labs at So Fi LAX it can't be used realtime. They have to have a human look at the Mata VR output Replay before putting it on the air.
There is little reason to think the timing had anything at all to do with Trump, nor is it the first open source LLM available to Russia or North Korea or anyone.
Partially agree. A year ago their algorithmic efficiencies were known but no one paid attention. But LLM's are not 03 or R1. This is a different universe where humans are no longer needed for training. The CCP is always working an angle.
Again, it's a matter of comparison and of opportunity costs compared with other conceivable models, including the one that China now seems to represent.
Of course US companies will continue to produce innovation, but they will do so according to the logic of the funding/investment model I described, which has some acute costs that go along with any innovation, not least of which is the extreme potential for rent-seeking against the interests of both consumers and potential competitors in innovation. (And it's a counterfactual, but we need to consider what rate and type of innovation and what social outcomes might have obtained these past 30-odd years under a different system, even one which did the minimum of systematically dismantling massive, anti-competitive firms before they had the ability to buy up and throttle their emerging competition).
And about the whole "Sputnik Moment" thing, there's a temptation to see Chinese AI as representing an a new "arms race". But this analogy is flawed. Rivalry with the old Soviet Union immediately took the form of an actual arms race (which turned into a global inter-imperialist rivalry) because the historical context was one in which both systems had brand new military industrial complexes (and in part because both society's understanding of the consequences of war were naive, being rooted the experience of WWII.)
US-China rivalry need not be this kind of zero-sum game-- that is, if the US responds by reforming its whole regime of national investment in order to successfully compete with Chinese innovation. The US's other alternatives are to: 1. Hope that the Chinese model will not continue to work in the long run; and/or 2. Attempt to deploy its residual economic, geostrategic, and military predominance to forcibly deny China the fruits of what might turn out to be its superior regime of late capitalist innovation.
There is no question that the world would be better off if, in the end, the US was able to tame the power of its monopolies and put the oligarchs who control them in their place in order to better compete with China economically rather than militarily, or by other extra-economic means. Choosing the latter course (around which, at the moment, there seems to be a bipartisan consensus) will lead to both pressure towards military conflict and retarded net economic global productivity and overall well being.
In many ways the fate of settled life as we know it depends largely on the ability of the US ruling class (in both its economic and political manifestations) to reform itself. And so far, the odds of this ever happening don't look great.
my post was more pointing out the dumb DEI complaining the other poster was doing. But still:
I have a hard time understanding the framing of “monopolies” in terms of US tech and AI research. There’s currently intense competition among 5+ US tech firms with many of them deploying $10s of billions of capital per year. Nothing about that is monopolistic. Cutting edge AI research requires huge amounts of capital so it’s naturally going to be done by the biggest companies — but smaller orgs can still find room to innovate and competez
This absolutely is an arms race. The use cases of advanced AI include weapons and war just as the use cases for orbital rockets did in the 50s.
China is quite clear in their goals to break up US hegemony and overtake them. There’s nothing wrong with chip bans and other policies if you think the US staying ahead in the AI race is valuable. It think it is - I think you have to pragmatically assess China as a rival and avoid fairy tales of de-escalation and cooperation.
Lastly, I don’t see the evidence for the “Chinese model” being better than the US research system. Certainly DeepSeek is impressive and close to parity, but the bulk of key advancements have still been made by American research teams. They’re certainly competitive but saying “this is a sign the US needs to rethink its whole economic structure” is hyperbole.
Point taken about the continued existence of a degree of competition within US tech. But I'm far from alone in arguing that they are still too big AND too shareholder-oriented, which remains a recipe for rent-seeking and highly destabilizing investment bubbles driven by hype ahead of, or out of proportion to, the potential pay-off in terms of innovation. This is a thing no one should need to be reminded of after this week in particular, if they are too young to recall the late 1990s or the financial crash in 2008 (different industry and product, but the same logic at play). Markets will now gyrate wildly around AI investment for a while, which is hardly the most rational way to fund research and innovation in a field that is supposedly SO vital to future economic development.
But then there's the question of whether it actually IS vital and not just more hype designed to attract investment. This where a system that actually planned large scale investment around some conception of proven future human need, rather than vague ideas about what today's consumers may want, or may be convinced to want, would be far superior. I'm not saying China has that kind of system, but we know what they have is far closer to it than what the US currently has. In short, they don't have a class of massively wealthy private investors interested primarily in short term gains dictating large scale industrial investment from month to month (and sometimes from minute to minute!).
And my point about an "arms race" is that today, unlike the 1950s, no one is under the illusion that actual warfare between two global economic powers is desirable, or even thinkable, really. Even proxy wars using modern technology are unimaginably horrific and feared by any reasonable person today. You're undoubtedly right that AI will have significant military applications, but this will have very little to do with actually planning for war and everything to do with these giant companies rent-seeking through lucrative government contracts. Just like automobile and aeronautics companies before them, these companies just want fat cost-plus contracts in perpetuity, and probably fear the actual use of the weapons they help create as much as any of us. And of course the point is that this amounts to a very wasteful form of international competition, when it isn't actually catastrophic for settled human life.
This post was edited 3 minutes after it was posted.
Help us build the best running shoe review site for a chance to win a LetsRun t-shirt.Help us build the best running shoe review site for a chance to win one of 10 LetsRun t-shirts.
