Jeph Jacques wrote:
laughing so hard that Sam Altman needs 100 trillion dollars and 15000 coal burning power plants and all the IP humanity has ever produced to make AI work and then four dudes in China are like "we did it with a chromebook from 2009 and some pringles cans"
-
71
-
Uncle Bosie wrote:
Raddison wrote:
My reaction to this was is this a turning point?
Back when I was a kid "Made in Japan" meant garbage. Cheap, tacky garbage. Then in the 1970s it all changed. Suddenly Japanese cars, stereos, cameras, etc. became the benchmark for quality stuff. The West suddenly had to play catch-up. Major corporations announced they were adopting Japanese quality control methods, Japanese logistics etc.
I worry that the same thing is happening with China. Yes, for a long time "Made in China" meant crap. Cheap dollar store stuff. But I worked with Chinese companies and it was impressive how much they wanted to know about what they were making for us, why some defects could be accepted and why some could not.
Their approach was totally different to Western suppliers. Western companies just wanted to get paid. If we rejected parts they bitched and moaned and claimed the defects were impossible to fix or they would need to double the price etc. The Chinese wanted to work with us to improve it. However, it was also very clear that they were data mining us at every opportunity. We had to be very careful what we told them but it was obvious their long term intention was to replace us.
I can't help thinking this may be the start of a sea change in design and manufacturing. We used to say the Japanese can't make anything original, all they can do is make cheap copies. Today we say the same thing about the Chinese but we may find we are wrong about that as well.
Chinese are much smarter than Americans who are saddled with POC/DEI subgroups with lower IQ. China ranks near the top in IQ. Computing and even more so programming is all math. This shouldn't be surprising if they figured out a better way to AI than Sam Altman. Altman seems much more concerned with protecting Open AI's channels to make $. I read his company is burning through so much cash and needs so many more chips. If Deepseek is really good as it is, Altman has failed. Have you seen AI work in Google search and customer service. It's garbage now.
I actually thought it might take longer for someone offer a Ramaswamian cultural and political take on this-- i.e. the Chinese don't do "woke" and DEI and have higher I.Qs.
But the relevant differences between the US and China when it comes to innovation are historical and structural. Chinese capitalism is based on a planning model and US capitalism is increasingly based on a private, monopolistic, shareholder-centric model.
Not enough people listened to Lina Khan when she said that corporate concentration in the US, and in tech in particular, was bad for US consumers and tax payers, but also for US capitalism itself as a driver or innovation.
China has been using its historically specific advantages-- a unitary state and a one party system-- to plan investment in pursuit of a national catch-up in strategically important specific industries. Instead of allowing the fruits of growth to be siphoned off by a class of mostly superrich shareholders, it plows them back as reinvestment for a predetermined period of time (typically, 5 years). This might simply turn out to be a superior way of doing capitalism in the 21st C, whatever one might think of the Chinese system as a whole. And before anyone chimes in with stories about failed Soviet planning, consider what a difference it makes now that there is advanced IT and telecom. Simple barcoding, never mind data harvesting, produces massive amounts of highly detailed real-time data that can inform the allocation of resources to meet actual demand in an economy.
And with the US well on its way to post-liberal domination, oligarchy, and aggressive isolationism, it may no longer be able to tout its unique social and political freedoms versus China OR its innovativeness.
According to its own story, capitalism is supposed to create products that improve human life by solving material problems. But is seems that US capitalism increasingly functions to make a few individuals unimaginably wealthy while avoiding the kind of competition that spurs innovation. Meanwhile, China seems to be making products that actually solve current and future problems-- cars, green tech, and now, apparently, AI-- and making them affordable to boot.
This post was edited 1 minute after it was posted.163 -
Here's a good article that shows that the big breakthrough was from innovations they had to try to work around a constraint (H100 chip ban). That allowed them to train a LLM for way cheaper than the others.
Ben Thompson wrote:
Here’s the thing: a huge number of the innovations I explained above are about overcoming the lack of memory bandwidth implied in using H800s instead of H100s. Moreover, if you actually did the math on the previous question, you would realize that DeepSeek actually had an excess of computing; that’s because DeepSeek actually programmed 20 of the 132 processing units on each H800 specifically to manage cross-chip communications. This is actually impossible to do in CUDA. DeepSeek engineers had to drop down to PTX, a low-level instruction set for Nvidia GPUs that is basically like assembly language. This is an insane level of optimization that only makes sense if you are using H800s.
Meanwhile, DeepSeek also makes their models available for inference: that requires a whole bunch of GPUs above-and-beyond whatever was used for training.
So was this a violation of the chip ban?
Nope. H100s were prohibited by the chip ban, but not H800s. Everyone assumed that training leading edge models required more interchip memory bandwidth, but that is exactly what DeepSeek optimized both their model structure and infrastructure around.
Again, just to emphasize this point, all of the decisions DeepSeek made in the design of this model only make sense if you are constrained to the H800; if DeepSeek had access to H100s, they probably would have used a larger training cluster with much fewer optimizations specifically focused on overcoming the lack of bandwidth.50 -
wejo wrote:
Got a few questions about deepseek.
TLDR: 1) Do you trust the claims being made 2) Isn't this a good thing long term?
1. Not a chance. Early stage tech startups spend $5 million a year with engineering teams of 10 people or fewer to work on way less demanding type products. Something like Deepseek takes a way larger budget just to fund the R&D team.
The model is said to have 671 BILLION parameters being trained. Parameters are literally numbers that get recalibrated during backpropagation, the learning phase. And that is just training the multilayer perceptron part of it. There are further parts like vocabulary embeddings that use an "Attention" model and are learned with parallel processing. The hardware to do this is incredibly expensive, needs to run very fast, support parallel algorithms (cuda cores, tensor cores), needs to hold vast sets of data.
The training data and data pipelines require enormous amounts of work. To make it respond like a human, it should learn from as much data as possible, which is essentially training it on nearly ALL the data available on the internet. All that data needs to be prepped into formats the model can use and "ingest" for training.
The model itself is not the entire software. It needs a framework for the model to operate with users. The conversational "memory", prompting, output formatting, tone/mood, censoring, and so on are different layers of software, not the model. That also costs a fortune to build and maintain.2) It depends on who is asked. Idealists may say yes, arms-race theorists may say no, etc etc. You get the idea. Either way, it is very doubtful the open source software has any true breakthroughs in its modeling. The big breakthroughs like Convolutional Neural Nets or Transformers with Attention seem to be few and far between. If Deepseek has a major breakthrough, then great, the breakthrough is visible worldwide.
24 -
And the fast that it's Open Source means that you, I, and any of Ai Big Name Companies can get an exact copy of the changes to the code in Deep Seek's github repository.
11 -
Uncle Bosie wrote:
It's not Chinese claims. You can download the app. It's #1 download in Apple store. The AI experts who tried Deepseek said it was better than Open AI. A company doesn't lose $600 billion market value for nothing.
Well, Meta kind of did in 2022, and then quintupled in value in the past two years or so. There is real cause for concern for the dominance of US AI, but I'm not yet convinced that Nvidia's profits will slow any time soon. I'm definitely in high alert, however.
30 -
SB wrote:
Not enough people listened to Lina Khan when she said that corporate concentration in the US, and in tech in particular, was bad for US consumers and tax payers, but also for US capitalism itself as a driver or innovation.
I partially agree with this but there are some notable exceptions. Bell Labs was the R&D arm of a telephone monopoly which invented all sorts of incredible things in large part because of their monopolistic position.
I think Google and Facebook had the opportunity to do something similar but their "inventions" were DEI and ESG projects and thus will have near-zero long term technological impact.
If you actually think about Google, they are basically still a search engine that got better at advertising. And if you think about Facebook, they're still a social network that also just got better at advertising (I personally think their primary product, a social network, has gotten significantly worse over the past two decades).
The opposite of Google/Facebook would be someone like Elon Musk who astonishingly at every turn finds a way to push something new.
We need monopolistic powers for innovators like Musk and anti-trust action against low-innovation companies like Google/Facebook.
The Lina Khan's of the world, with their hard-left, DEI/ESG progressive beliefs, should not be in positions of power to be making these decisions.
514 -
XDFXDFXDF wrote:
Yes, I think it is a good thing, unless your portfolio is heavy on NASDAQ stocks.
That's a big if. Maybe one portfolio out of ten isn't heavy on NASDAQ stocks after the decade-long explosion of tech valuations.
00 -
the count of monte carlo wrote:
wejo wrote:
Got a few questions about deepseek.
TLDR: 1) Do you trust the claims being made 2) Isn't this a good thing long term?
1. Not a chance. Early stage tech startups spend $5 million a year with engineering teams of 10 people or fewer to work on way less demanding type products. Something like Deepseek takes a way larger budget just to fund the R&D team.
I'm not buying it either - China loves lying about things.
If anything, I think it's a scare-tactic to get Trump/Americans willing to spend more on America-AI related companies. The timing was odd and the unison of the narrative has my spidey-sense on full alert.
Over sold market today. I bought calls.
09 -
Flush Twice! wrote:
NASDAQ is going to get hit again tomorrow morning! DeepSeek just dropped their AI image creation tool about and hour ago and people are already saying it is superior to OpenSI Dall-E 3.
Going to be some very stressed out VC folks tomorrow.
Interesting news and a reasonable fear, but for now, anyway, futures markets are tilting toward a verdict of "no need for further panic."
00 -
krispy kremlin._._._._._ wrote:
SB wrote:
I partially agree with this but there are some notable exceptions. Bell Labs was the R&D arm of a telephone monopoly which invented all sorts of incredible things in large part because of their monopolistic position.
I think Google and Facebook had the opportunity to do something similar but their "inventions" were DEI and ESG projects and thus will have near-zero long term technological impact.
If you actually think about Google, they are basically still a search engine that got better at advertising. And if you think about Facebook, they're still a social network that also just got better at advertising (I personally think their primary product, a social network, has gotten significantly worse over the past two decades).
The opposite of Google/Facebook would be someone like Elon Musk who astonishingly at every turn finds a way to push something new.
We need monopolistic powers for innovators like Musk and anti-trust action against low-innovation companies like Google/Facebook.
The Lina Khan's of the world, with their hard-left, DEI/ESG progressive beliefs, should not be in positions of power to be making these decisions.
Understand your point but Google had made self driving cars that are the best in the business right now.
41 -
krispy kremlin._._._._._ wrote:
the count of monte carlo wrote:
1. Not a chance. Early stage tech startups spend $5 million a year with engineering teams of 10 people or fewer to work on way less demanding type products. Something like Deepseek takes a way larger budget just to fund the R&D team.
I'm not buying it either - China loves lying about things.
If anything, I think it's a scare-tactic to get Trump/Americans willing to spend more on America-AI related companies. The timing was odd and the unison of the narrative has my spidey-sense on full alert.
Over sold market today. I bought calls.
$5 million total is false. It's obvious the people who say it's true know little to nothing about AIDL or software engineering. It's like telling a non runner someone ran a 2 minute 5k. The non runner is oblivious.
A well trained pretrained model (the P in GPT for example) alone costs significantly more than $5 million in R&D. For the deluded people out there, that is just the next word predictor, the model before the model. A multilayered feed forward neural net outputs a probability distribution of the words in its vocabulary based on the previous words. This type of model is not new, has been around for decades, and easily surpasses that fictitious $5 million number.
41 -
the count of monte carlo wrote:
$5 million total is false. It's obvious the people who say it's true know little to nothing about AIDL or software engineering. It's like telling a non runner someone ran a 2 minute 5k. The non runner is oblivious.
A well trained pretrained model (the P in GPT for example) alone costs significantly more than $5 million in R&D. For the deluded people out there, that is just the next word predictor, the model before the model. A multilayered feed forward neural net outputs a probability distribution of the words in its vocabulary based on the previous words. This type of model is not new, has been around for decades, and easily surpasses that fictitious $5 million number.
That Ben Thompson DeepSeek FAQ (post #43 above) addresses that:
Ben Thompson wrote:
I’m not sure I understood any of that.
The key implications of these breakthroughs — and the part you need to understand — only became apparent with V3, which added a new approach to load balancing (further reducing communications overhead) and multi-token prediction in training (further densifying each training step, again reducing overhead): V3 was shockingly cheap to train. DeepSeek claimed the model training took 2,788 thousand H800 GPU hours, which, at a cost of $2/GPU hour, comes out to a mere $5.576 million.
That seems impossibly low.
DeepSeek is clear that these costs are only for the final training run, and exclude all other expenses; from the V3 paper:
Lastly, we emphasize again the economical training costs of DeepSeek-V3, summarized in Table 1, achieved through our optimized co-design of algorithms, frameworks, and hardware. During the pre-training stage, training DeepSeek-V3 on each trillion tokens requires only 180K H800 GPU hours, i.e., 3.7 days on our cluster with 2048 H800 GPUs. Consequently, our pre- training stage is completed in less than two months and costs 2664K GPU hours. Combined with 119K GPU hours for the context length extension and 5K GPU hours for post-training, DeepSeek-V3 costs only 2.788M GPU hours for its full training. Assuming the rental price of the H800 GPU is $2 per GPU hour, our total training costs amount to only $5.576M. Note that the aforementioned costs include only the official training of DeepSeek-V3, excluding the costs associated with prior research and ablation experiments on architectures, algorithms, or data.
So no, you can’t replicate DeepSeek the company for $5.576 million.
I still don’t believe that number.
Actually, the burden of proof is on the doubters, at least once you understand the V3 architecture. Remember that bit about DeepSeekMoE: V3 has 671 billion parameters, but only 37 billion parameters in the active expert are computed per token; this equates to 333.3 billion FLOPs of compute per token. Here I should mention another DeepSeek innovation: while parameters were stored with BF16 or FP32 precision, they were reduced to FP8 precision for calculations; 2048 H800 GPUs have a capacity of 3.97 exoflops, i.e. 3.97 billion billion FLOPS. The training set, meanwhile, consisted of 14.8 trillion tokens; once you do all of the math it becomes apparent that 2.8 million H800 hours is sufficient for training V3. Again, this was just the final run, not the total cost, but it’s a plausible number.40 -
wejo wrote:
Got a few questions about deepseek.
1) How do we know they actually built for $5 million what others spent billions on?
But apart from that if a) AI is a good thing and b) Deepseek shows AI can be built at like 1/100th, or 1/10th of the cost and c) is open source - isn't that a good thing?
A lot of people won't trust a Chinese company with sensitive data but isn't this an open source model that anyone can use right?
Maybe a little explanation on the open source part but at the very least someone can adapt the code for their own uses right?
If I told you 2 weeks ago, "hey there will be this new AI model, it cost about 1/50th of the existing model and uses a ton less power." Everyone would have said "great"?
TLDR: 1) Do you trust the claims being made 2) Isn't this a good thing long term?
1. We don't. As Flavor Flav said Dont Believe the Hype.
2. Ask deepseek about Winnie the Pooh or Xi Jinping and see what it says.
10 -
Downloaded the app. It sucks and pretty sure it put malware on my phone. AI continues to under impress
25 -
I agree with this post.
For years we've heard how US math scores lag most countries, while China leads.
For years we've outsourced our manufacturing to China to save cost.
Recently we've heard that China is wildly outpacing US in AI patents and research.
I don't know if Deepseek claims are legit, but if it's not Deepseek it will be someone else coming out of China.
The US was once a leader in robotics, semiconductors, 3D printing, drones, etc. We developed this tech, then the investor class squeezed those companies to maximize short-term profit, for long-term innovation/growth.
I work in automation/robotics and touch on AI. It's alarming how much better their tech has become. I recently bought a $1K Bambu printer, it runs circles around my $50K Stratasys. It's not even close. Stratasys - a US company - literally invented the tech but China has perfected it. Its sad.
100 -
(sorry, meant to hit reply to the earlier post that mentioned Made in Japan used to be low quality but eventually transformed into tech leadership)
10 -
here's the github repo
31 -
heres the huggingface repo:
11 -
here's the website:
11
What People Are Talking About On LetsRun
No top threads at the moment. Check back soon.