AI Expert Deep Dives NVIDIA vs. Tesla's DOJO

Enjoy this clip? Watch the full video here:

If you'd like to support the channel, you can find me on Patreon here:

Shopping at Tesla? You can support the channel & save by using my Referral code:

PANELIST INFO:

James Douma

Farzad Mesbahi
YouTube - @farzyness

CHANNEL LINKS:
Hans' YouTube channel - @HansCNelson

#Tesla #elon #musk #elonmusk #teslaengineering #teslaai #teslafsd #fsd #fsdbeta #autonomousvehicles #autonomy #neuralnetworks #machinelearning #engineering #deepmind #nvidia #dojo #teslanews

Tesla, Tesla Stock, Elon, Musk, Elon Musk, Tesla Engineering, Tesla AI, Tesla Energy, Tesla Manufacturing, Tesla Stock Videos, Tesla Dojo, AI, AGI, LLM, LLMs, Technology, Software, Computing, machine learnings, AI models, Engineering, Tesla Bot, Bot, Optimus, Optimus Gen 2, Tesla Optimus

If NVIDIA's HATES Competition, What About DOJO?
AI Expert Deep Dives NVIDIA vs. Tesla's DOJO

[AFeigenbaum]

Watched the full video right after it came out ... astoundingly informative ... more please ...

[fredpsimas]

Exaflops of compute is part of the equation… data is another… power usage of your neural net is another…. Running sold robots like Tesla cars is another… ability to train is another… AND…Tesla has done it!

[XXfea]

This is the definition of GEAT info . . many thanks Hans!!

[Ronnieleec]

Nvidia confirmed that Tesla has committed to buy Blackwell.

[ronaldfradet]

It was a very interesting chat i am always amazed at James talking points❤

[Cybertruck]

I prefer this type of conversation rather than those that come over as blatantly Tesla fanboy. This is a bit more technical and a bit more real. I am a fan of Tesla btw.

[ae]

The interesting thing is that data centre in New York is all dojo, there is a hint there that Dojo is just fine.

[AronGari]

I think there is an interesting point that a lot of people are considering when comparing Nvida's Blackwell to Dojo chips. That is Tesla's dojo chips are application specific integrated circuit (ASIC) vs graphics processing units (GPUs) in the case of Blackwell. Obviously Nvidia has built out their GPUs to target Ai/ML workloads however a GPU is still general purpose and an ASIC is far more efficient and performant in the tasks it was specifically designed for then GPUs. My guess is Tesla will be using the GPUs they buy for data processing, software development, running Tesla's manufacturing management Ai, and the bulk of FSB training will be on Dojo. People should check out the Groqcard accelerator if the are interested in seeing the difference an ASIC can make in Ai workloads. As one last note the Groq accelerator uses 14nm (3 gens old) vs 4np (advanced 5nm) and it still very impressive.

[markkuskin]

Thanks Hans. This reminds of the difference between Theory and Reality. In Theory, Tesla is an AI company. But in Reality, Nvidia is.

[lukehaswell]

Intel and AMD will definitely have their share of the market. TSMC is at max capacity and investing in other semiconductor companies will be an absolute power move, I keep increasing my shares manageably. Different chips are good at different things and Nvidia has been very specialised, which leaves other aspects of Al open.

[mikmop]

As corrected by a subsequent post, a Blu-ray is 25 GBytes, or 200 Gbits. So at 192 gigabits per second, it's downloading about 1 Blu-ray per second or about 5 DVDs per second.

[KahnSkins]

Something that needs to be said about AI Day and the Nvidia presentations. They almost mirrored each other! Each started with the base line chip and scaled all the way up to cabinets from there! It was eerily similar. Did Nvidia take a page out of Teslas presentation? They even bragged about having the worlds first exaflop cabinet, when we know DOJO was presented as ten cabinets per 1.1 exaflop. It almost felt like Nvidia was competing with Tesla when they did this.

[Urgelt]

My opinion, unsupported by facts:

Tesla intended to make Dojo its main training compute asset. As much effort as they put into it, they had priority.

Two things probably happened.

Dojo maturity slipped, either because programming it took longer than expected, or hardware manufacturing slipped.

And nVidia GPUs, while not as optimized for the workload, got better. And we're available in bulk.

So I don't see Dojo as originally intended as an insurance policy. Instead, I see Tesla as pragmatic and flexible. They went and got the compute cycles they needed.

But for sure, Dojo is *now* an insurance policy. Or a fallback.

nVidia's Blackwell architecture is coming. Much better compute per watt than H-100. Tesla will probably stuff Blackwell into its data centers as fast as they can buy them.

And keep building Dojo, and work on Dojo 2.0.

It's nice to be in a strong cash position.

[DG-wofx]

I agree with you that the interconnect is the secret sauce. Physical and routing astounding amounts of data. The shuffle in Map/shuffle/Reduce.
I am looking forward to learning more about it. Intro to Dojo, chapter 2? 😅

[fredhearty]

With the demand for NVIDIA processors exceeding supply, and the commensurate high prices, it seems that there is a huge opportunity for offering AI compute-as-a-service to the masses of users that cannot buy a 10, 000 processor cluster. (Also, there are limited number of locations that can supply the electrical power for such a facility.) So, the opportunity for Tesla or someone to sell this service exists big time -- will be interesting to see who steps up to meet the demand.

[AndreasWest]

You mentioned at the start of the video that you talked about the limits of inference by the onboard HW. Does anyone have a timestamp/link to it? Would like to hear as I think right now that’s the biggest constraint with HW3. Thanks

[stevedowler]

So the competitors that Tesla's DOJO should look at are also working on 4 or 5 applications instead of thousands like Nvidia? Those companies are their competition if they are interested in competing. But if Tesla has chosen the right mix of applications then they are already ahead of the competition in that mix and they are accelerating in that space.

[steveclauter]

Although compute and software are big components to neural nets but I'd also add memory to the "chips" since these matrices are all built in memory before they can be processed. Plus how do you increase the processing power? Compute, software AND memory requirements need to be expanded!

[richardhamilton-gibbs]

James, I love your content. I'm going to expand this discussion to where Tesla goes next.

First, Tesla isn't avoiding talking about their D1 chip and Dojo. They're just not publicizing their progress.

Your comment: The three architectures required for AGI are CPU, GPU, and neural processors. Neural processors are central to AI architecture. GPUs and CPUs are peripheral.

AMD challenged Intel in CPU architecture and won that battle. In 2006, AMD purchased ATI Technologies, who manufactured GPUs. (Personally, I liked the Radeon boards better than the nVidia product.)

Tesla and AMD should be excellent partners. AMD can bring superb CPU and GPU design to the partnership, and Tesla can bring the best neural processing architecture in the world. Tesla also has the most advanced AI software in the world. The biggest AI problem ever addressed is autonomous driving and Tesla is about 6 generations ahead of the rest of the field.

I can see AMD and Tesla creating a new company together. It will be created by Tesla because this new company will need the Tesla Agile corporate structure. Honesty is the key.

That doesn't leave Nvidia out of the mix. Jensen and Elon have a huge respect for each other, and Elon will want Nvidia to join an American world-beating enterprise.

I'm a kiwi. In my eyes, American is a sociopathic capitalist culture. That's why newspapers could write about Tesla-killers as though business is all a fight to the death. I built a tech company here in the US. I've hired lots of Americans. I know Americans don't look at health care as a right, they don't care for their elderly or sick, so I implemented health insurance for my employees that covered their entire families, and long-term disabililty insurance to help them through tough times.

So Elon and I share cultural values. It isn't all about sociopathological capitaslism. It's about altruism, honesty, and collaboration. Elon has given you the clues again and again and again but most Americans still can't hear the message. What Elon wants is for everybody to be successful together, to work towards a good life for everybody. Like, Americans are all amazed that he hasn't gone off to live on his own private island somewhere, with the American sociopathic "I've got mine" mentality.

Wake up, America. Elon is lighting the way to a new philosophy.

[giansolomon]

Incredible timely discussion thanks guys