TINY 70 watt RTX 4000 SFF ADA Generation GPU For AI, Docker, Plex, Jellyfin, and MORE!

Check out the Ubuntu/Ollama Forum thread here:

How could something so small be this big of a deal?! Well, Wendell is here to show you! Follow along with Wendell as he sets up his RTX 4000 SFF ADA generation GPU from Nividia for a streamlined AI experience, a home server beast, and whatever else your creative mind can come up with!

0:00 Introduction
1:26 Background and Use Case
2:52 VS the 4090?
4:06 RTX 4000 Specifics
5:00 Software Setup
6:12 Step by Step
17:12 Wendell Reflection Time
18:24 Results and Issues
28:24 Conclusions

********************************
Check us out online at the following places!

-------------------------------------------------------------------------------------------------------------
Music: "Earth Bound" by Slynk

Edited by Autumn

[Maximum_Nova]

1. Take the cover off the gpu,
2. plug the little fan into a fan extension cable and route cable out the cover as you reinstall the cover
3. plug extension cable into mobo after you put the card back in the computer
4. ~5 watts unlocked. 5/70 is over 7%, so you can get a nice little boost on the card's performance freeing up that bit of the power envelope

[CraftComputing]

4000 ADA! Literally my favorite GPU of the last few years. Glad you got one to play with finally 🙂

[mokahless]

It's an older meme, sir, but it checks out

[JoeBlow]

Just a minor point? The application 'nvtop' is not Nvidia top. It's 'Neat Videocard Top' as it works on all GPUs. Everyone just assumes it's nvidia top but it's not.

[bamtoday]

RTX 4000 ADA + Minisforum BD790i = the dreamiest ITX build combination.

[SuperMari]

Got an A2000 16gb on the way for my trans-coding galore, can't wait to tinker with it. Wendell dropping appropriately timed videos as usual.

[maximilianott]

I don't understand half of what wendell is saying...but my god, its well spoken, depp techtalk, and i'm a simple guy... i see a new wendell video, i click on it, i watch it

[Seandotcom]

I have the RTX A2000 12GB for similar purpose. It’s less performant in basically every regard but also much cheaper and can easily chill in a 2U chassis :)

[davysprocket]

That m.2 being held in by the pressure of the gpu made me chuckle :)

[TavishMcEwen]

Note that NVTop stands for "Neat Video TOP", not "NVidia TOP"
It works on AMD and NVIDIA, and recently got support for Intel!

[Maxjoker]

7:16 Props to Wendell for calling out the incredible danger of curl piped into SH. Did you know, for example, that a server can often tell via side-channels if you just curl a command, or if you curl it into a shell? It's very possible to create a downloadable URL that delivers one script if you just curl to stdout/a file, and another if you curl pipe bash. Always download shell scripts, review them, *then* run them!

[YoungSecurity]

I help contribute to open-webui, and we recently refactored the Dockerfile and added the feature to include ollama using build args and build the container for GPU accelerated embedding. So you don't have to install Ollama in a separate container, unless you require that.

[NickAubert]

Another interesting aspect of this lower power usage is that you could install several of them into a workstation without having to seek out specialist power supplies. Four of these would come to 80 gigs of VRAM at just 280 watts. Too bad they're so expensive.

[mauquibzonedout]

I can only afford the RTX 2000 ADA. The RTX 4000 ADA seem to be a pretty good option. Good video!

[EduardoReyesDPM]

Awesome video like always but you are just teasing us with $1500 hardware, For the next video please do a ARC 770 video showing how to enable SRIOV!

[CaptTerrific]

Factor in the external watercooling I need for my dual 4090s, and I'm really regretting not having had this card as an option when I built :D

[icey_u]

i cant wait to pick this up in a few years for my plex box

[rushunt]

I own one, for a small pc gaming build, its more powerful than my 2080ti its amazing.

[BigHeadClan]

Damn that's a pretty potent little professional class GPU, might be the same price on paper as a 4090 but you can pretty much guarantee it will work with any recent Server or Workstation without any concerns for new components like a PSU or generating crazy amount or simply fitting in your existing systems/taking up multiple PCI-E slots or risers.

So in that sense it actually makes a lot of sense over something like a 4090.

[allenmueller]

For years I have asked about better graphics cards for imaging, design, 3D, etc. I use a lot of apps, and I do things that aren’t normally done with graphics. Whenever I ask around on forums, I’m either flamed or given vague answers, always boiling down to “just get a gaming GPU”.

THIS video touches on some of the stuff I’ve hoped someone, somewhere, would cover.

I love my macs, but I’ve wanted to build a powerful machine so I can do more iterations, of higher quality, as fast as I can think of them. Fitting in SFF and not using a ton of wattage would be great too.

Thank you! More content like this please!