Optimizing TrueNAS for SPEED (and Safety)

After setting up your TrueNAS server there are lots of things to configure when it comes to tuning ZFS. From pools, to disk configuration, to cache to networking, backups and more. This guide will walk you through everything you should do after installing TrueNAS with a focus on speed, safety, and optimization.

📺 Watch Video

Disclosures:

• Nothing in this video was sponsored

Pool Optimizations

I chose Mirrored VDEVs for a few reasons:

Upsides

• Provides redundancy
• Increased performance
• Great for small random reads
• Expansion, much easier than any other raid type
• You can expand by adding an additional pair
  • This means I only need to buy 2 disks to expand my pool
  • Resilvering is fast

Downsides

• Write performance can be impacted since each write involves updating all VDEVS in a mirror
• Adding a separate ZIP like an SSD can improve write performance, which we’ll talk about
• If I lose 2 drives in the same mirror I lose the whole pool
  • Which means that I need to replace disks fast
  • If your pool of mirrored VDEVs is big enough, the chances are less likely to happen
• 50% capacity loss

Datasets

• Encryption: On
• Sync: Standard
• Compression: LZ4
• Enabled Atime: Off
• ZFS Deduplication: Off

• Adaptive Replacement Cache, this is where data is stored that is frequently accessed along with metadata about the files.
• Algorithm for adding and evicting data for least recently used.
• This makes accessing files quicker because it doesn’t need to read from disk and takes the load off those disks so they can do another tasks
• This isn’t really used for write, that’s ZIL, but some async writes and write directly to ARC
• There’s a general rule of thumb that you should have 1GB for each TB, however the more the better.
• One tweak that you might have to do if using ZFS on a Linux system like TrueNAS Scale and that’s adjusting the ARC allocation
  • This is not needed after this patch is released
  • By default linux only allocates 50%
  • You can override this with a quick command while the system is running and then you can also follow that up with a startup command to be sure that this persists between reboots
  • Generally speaking you should let your system run for a few days before making this adjustment as well as running any services and VMs to be sure you don’t over allocate your RAM to ARC.
  • See Tom Lawrence’s Video on this topic

Read Speeds

To increase read speeds:

• More RAM
• More disks / spindles
• L2ARC
  • Stripe these, even 1 large SSD is good

Write Speeds

To increase write speeds:

• More Disks - spinning disks can only read/write at about 150 MB/s
• Disable Sync write? (No!, not safe)
• Create SLOG device
  • Should use disks fast that in the pool like SSDs
  • Be sure you have at least a mirror for this
  • This moves ZIL off the main pool, which frees up the pool to do other things

Network Considerations

• LAGG / LACP gives you 2 lanes
• Gives you redundancy
• Generally speaking if your switch supports it, a great idea

Network and disk speed go hand in hand

• If your pool can’t read at 10 Gb/s there’s no sense of having 10 gb/s networking
• If your disks have huge IOSps (think all SSD) your network can’t handle the throughput
• Somewhere in the middle

VLAN Routing

• Another huge consideration is inter VLAN routing.
• If you have to cross VLANs, be sure that your switch or router can handle these speeds or you might have to get creative with a dedicated data network that doesn’t traverse VLANs.

Snapshots

• This is diff based
  • Fist will be relatively large
  • Subsequent will be smaller

My Settings:

• Lifetime of 2 weeks
• Enable Recursion
• Don’t allow taking empty snapshots
• Enable the schedule

Backups

• Yes, but hard to do on a budget
• Ideally you want to replicate these snapshots to another pool, preferably on another machine

Alerts

• Check to be sure alerts are enabled and working

• Yes, enough said

Where to Buy

Here are some items I found useful when building my NAS

(Affiliate links may be included. I may receive a small commission at no cost to you.)

Join the conversation

Over the last few weeks while building my new TrueNAS server I learned a lot about ZFS and how to optimize my new NAS for performance.https://t.co/qvAJBBSCkA pic.twitter.com/6LB5IqHZo7

— Techno Tim (@TechnoTimLive) February 9, 2024

Links

⚙️ See all the hardware I recommend at https://l.technotim.live/gear

🚀 Don’t forget to check out the 🚀Launchpad repo with all of the quick start source files

🤝 Support me and help keep this site ad-free!