Configuring TrueNAS NFS Share for XCP-ng

XCP-ng Networking Explained

iSCSI VS NFS

ZFS is a COW

Connecting With Us
---------------------------------------------------

Lawrence Systems Shirts and Swag
---------------------------------------------------

AFFILIATES & REFERRAL LINKS
---------------------------------------------------
Amazon Affiliate Store

UniFi Affiliate Link

All Of Our Affiliates that help us out and can get you discounts!

Gear we use on Kit

Use OfferCode LTSERVICES to get 10% off your order at

Digital Ocean Offer Code

HostiFi UniFi Cloud Hosting Service

Protect you privacy with a VPN from Private Internet Access

Patreon

Chapters
00:00 TrueNAS NFS Share for XCP-ng
01:49 Creating Dataset, Permissions and NFS Share
03:00 Add NFS Shared Storage in XCP-ng
03:45 ZFS and SYNC risks
06:14 NFS and IP Security
07:02 Speed Difference with SYNC
08:44 NFS and Dedicated Storage Network

[AlexDiamantopulo]

Exactly what I needed! I just asked you this question yesterday... WOW!
Thanks so much!

[jdmcivicrrr]

Thanks for making these TrueNAS Scale tutorials ❤

[GennPen]

As an option, in order not to disable SYNC in NFS, you can enable the cache on virtual disk (for example, Proxmox allows this). This gives better read and write performance.

[SyberPrepper]

Super helpful video Tom. Separate storage network is always high on my list. Thanks for the pros and cons of the NFS syncing.

[adam]

I've always used NFS for hypervisor storage sharing when the option has been available. It's simple and has thin provisioning out of the box. I've also never seen any performance issues with the production workloads. The convenience and supportability of NFS has over time been more beneficial than any performance gains iSCSI or other methods have provided, in my experience.

[sinisterpisces]

This is an awesome video. I know it's aimed at XCP-ng users, but the TrueNAS side of things is a great starting point as well.
In particular, TrueNAS Scale, as of Wednesday, June 26, 2024, doesn't make it obvious in the UI that the generic template is the best dataset template for NFS use. Since NFS is meant for *nix filesystems and the generic permissions are generic unix permissions, it's implied, but if you don't know that, it's not super-clear. One of those little details that makes the learning curve a bit steeper than it needs to be, but that will hopefully be fixed with a minor adjustment to the UI someday. :)

Though, I'm curious about one point: was there a reason you avoided using ACLs for the NFS share?

[xtornado]

Recently i was helping my friend to set up truenas enterprise that they company bought.

It is connected to two switches that running mlag between them over 2x40G qsfp (mikrotik CRS326-24S+2Q+RM) yes mikrotik and works excellent for holding l2 over mlag.

On truenas was bond 2X10G to each of switch and create 4 vlan tags over that:
-mgmt vlan
-nfs vlan <-> proxmox for vm storage
-nfs vlan <-> vm nsf shares for db puroposes (vm is only boot drive and nfs mount is for db data and logging) with optimized block sizes
-smb vlan

And have restrictions for each segment per ip, and fw isolation between vlan so nothing cannot talk with each other.

Was do not noticed any slowdowns regarding additional tagging.

Running all on 1500 MTU but will try jumbo frame scenario also.

Truenas backup server for replication is ongoing process, it will be truenas scale vm with HBA
passthrough.

Was setup this after inspirations from your truenas video series, thank you for all of that

You are true truenas ambasador :)

[MrDukeLeto]

Hi Tom! Thanks for a new great video and your contribution to the popularity of XCP-ng. When you showed your network adapters attached to the storage network, I noticed that the MTU is set to 1500 bytes. Did you try to set it to 9000 bytes? Was there some performance advantage? Sorry if you already answered to this question in another video.

[darrellpatenaude]

Thanks I was wondering how to do this. I'll try this when I get home from work

[FTLN]

Hey Tom, why so slow with NFS Sync disabled, shouldnt you be getting close to wire speed ?

[lorenzomainini]

Great Timing! i was just about to do something like that. I deployed a Trunas and I want to create 2 dataset: 1 for xcpng vm storage to set HA and one for the file share of the AD to manage with the file server role on windows server.
Do you have anything to suggest for the latter?

When adding the NFS share to xcpng though it returns a "SR_BACKEND_FAILURE_73(, NFS mount error [opterr=mount failed with return code 32], )" error, i think i gotta dig deep.

EDIT: I thought it was going to be harder, for TrueNAS core apparently the "All dirs" checkbox in the NFS share settings has to be selected because xcpng like to mount subdirs.

[rudypieplenbosch]

Interesting, i never considered this 🤔

[bzmrgonz]

Lawrence, what about minIO in the mix.. please do a 3 box setup talk buddy (xcp-ng + truenas + minIO ). I think this is a solid enterprise level core setup.

[mspencerl]

NFS can either have synchronous or asynchronous writes via the NFS client.

It seems not all softwares/ vendors have the same defaults so there's a chance whatever NFS client is writing to an NFS share can also have sync or a sync enabled or disabled.

Do you know what XCPNG does by default? If it does synchronous by default then it may not even be necessary on the NAS

[darrellpatenaude]

Running Xen Orchestra on Truenas Scale app, has the same IP as truenas scale and I can't connect NFS Share because of same IP

[zyghom]

if you use compress on the dataset if=/dev/zero will make the writes almost non existent (probably better would be /dev/random)- my dd shows 3GB/s on SSD pool (dev/random shows 260MB/s) and 1GB/s on HDD pool (dev/random shows 250MB/s),
but what do I know ;-)

[succubiuseisspin]

Is there any way to build an active-active-HA setup for VM Storage with TrueNAS?

[James-xgjr]

Worth noting that XO should also have a NIC on the storage network.

[TheChadXperience]

MAC or IP filtering are but a mechanism to prevent automated systems from connecting to resources you don't want them to. It's so trivial to circumvent, and you shouldn't consider it to be any form of security feature, at all. It's not even worth considering. If you want security, you need authentication, which you simply can't do with NFS. NFS is not secure, and you should never use it over a hostile network. It's best to think of NFS and iSCSI like direct attached storage, but over a network. It has zero protection from attack. Anybody can easily spoof a MAC, IP or User ID. Especially, when it's all transmitted in the plaintext. It's like hiding your SSID. These things are to prevent machines from talking to things you don't want them to, unless they are purposely programmed to do so. They are not secure.