FASTEST Server Networking 64-Port 400GbE Switch Time!

We built a setup to get two servers attached to a $55,000 switch to see just how fast and cutting edge having a 64-port 400GbE switch is. If you are a networking enthusiast, this is the video for you. #networking

----------------------------------------------------------------------
Become a STH YT Member and Support Us
----------------------------------------------------------------------

----------------------------------------------------------------------
Where to Find STH
----------------------------------------------------------------------

----------------------------------------------------------------------
Where to Find The Unit We Purchased
Note we may earn a small commission if you use these links to purchase a product through them.
----------------------------------------------------------------------
- FS.com for the Switch
- PNY for the Mellanox/ NVIDIA ConnectX-7 cards

----------------------------------------------------------------------
Timestamps
----------------------------------------------------------------------
00:00 Introduction
02:29 External Hardware Overview of the 64-port 400GbE Switch
04:31 Inside a 400GbE switch with a Broadcom Tomahawk 4 Switch Chip
07:40 OSFP, QSFP-DD, and Challenges with 400GbE
13:15 Key Lessons Learned
15:36 Wrap-up

----------------------------------------------------------------------
Other STH Content Mentioned in this Video
----------------------------------------------------------------------

[josephesposito]

"This was a bad idea and It just keep happening, I don’t know what to tell you." Love it and can certainly relate. I'm glad you can get in trouble so I don't have to.

[edc]

Looks like a perfect switch for the pool house

[rush]

At work we are putting together two vmware clusters using a pair of Aruba 9300 switches (32 ports of 400G) as the backbone. vSAN storage with all NVME is going to love the bandwidth.

[TheChadXperience]

What a BEAST! I'm glad that you guys went and showed this off. I got trolled by a guy suggesting that I use infiniband (which I'm already familiar with) at home on one of my recent comments. I also understand that it is a total BEAR to set up and get running at the correct speeds, etc. Totally NOT the sort of solution I'd be considering for home use. I've seen other YouTubers try to install old used enterprise infiniband gear in their home, and it's always funny to watch them suffer. Too bad you didn't show all the fun bits of yourselves wrestling the bear. Although, it's not very exciting for non-nerds to watch. I'm surpised you could even power this thing. You'd need to build an internal Chernobyl just to spin up the fans, alone! (Exageration intentional for comedic effect) I'm afraid the electric company would send an investigator if I installed this in my home.

[Gabi-ctsz]

Mikrotik tomorrow morning be like: This is our CRS440-6QDD-1G-RM new 400 Gbps switch at just 2000$. Harmful joke, we love our Mikrotik friends and hopefully they love us back <3

[LSUEngineer]

Just more awesome, cool content from Patrick K & STH. Yes it's nerdy. Of course it is! We learned about Mikrotik's 4 port x 10Gb switch here from STH years ago and other OEM 25Gb & 100 Gb switches, too. Where else can you see an FS 64 Port x 400GB switch under the hood. You can count on Patrick. He will remove tell you how many screws it takes to get inside the device. And PK will remove the Heat sink, the CPU, the Fan Modules & PS Modules every time. And you get discussion, use ideas, performance results, connections parts, etc. It doesn't get better than that. STH just helps get my Nerd Habit filled each week along with the other online tech nerd content. Keep up the good work STH!

[nzalog]

Hey thinking of upgrading my home network, will this switch be enough for 1.2Gb Comcast? I don't want to bottle neck it, I'm paying big money each month.

[AaronPace]

At my work, we are refreshing a dc and where 400gig fits for us is more in the spine to super spine/core layer. We just don’t have hosts doing that high of speed, but most are moving from multiple 10gig connections to 25s. I would be jealous when there is a need for direct host 400gig!

[jasper]

I love you breakdown a switch in parts. Now I understand how it works. And yes 400gb is crazy. But 20years ago 1Gb in a server was crazy to. Thanks for a peak in the further 😎👍

[BarneyKB]

I love videos like this. Absolutely crazy. One thing I've always wondered: We hear about the huge undersea cables with huge bandwidths. How does the data get into these cables? Is it stuff like this? Would be cool to see.

[Buciasda]

I hope I will live long enough to get a 400Gbps Internet connection as a home user.

[DrivingWithJake]

Gotta love these we just picked up one of their 32x100g switches to play around with.

[legominimovieproductions]

Finally some reasonable connection for my truenas at home

[sarhtaq]

Sure love to see this kind of videoes.
As one who works at a school with 1G access and 10G between Spine/Leaf, speeds like this is unoptanium for us right now ;)

[__--JY-Moe--__]

super presentation Patrick! the server space is really moving fast! I'm glad U didn't show us a server bubbling in a cooling vat! but it sounds like this one gets warm.
I hope some1 has some winning cooling strategies out there! good luck Pat..

[chrish]

love the sharpie marks on the socket screws. Did you find a torque spec for the switch-chip socket or did you

[popquizzz]

Being a network guy, I am struggling with the OSFP acronym used here and i am sure when i start ordering these types of connectors/cables for these applications i will be corrected many times in conversation that overlap the two technologies. My one saving grace, 400Gb switching i will certainly see a lot of, 400Gb routing, I'm not so sure I will see a lot of this before I retire in another 10 years

[jk-mmto]

ServeTheHome are you at the Gates' home?

[threepe]

I wish I had found this channel so much earlier. I really find your gear overviews entertaining and informative. It's obvious you love what you do, and that sort of enthusiasm is contagious.

Even though i will never be able to justify a 60k switch at home, you've provided a whole ton of ideas and concepts that have been useful both at home and at work.

Selfishly seeking out your videos focused on more affordable gear though haha

[computersales]

Something I think would be interesting to see is the cost difference of being bleeding edge back in the 100Gb days vs now with 400Gb. I know $60, 000 is a lot of money but at the same time it seems cheap for what you were getting and what you could do with it if you have the ability to utilize it.