A collection of FIO test files for testing performance on various drives / arrays.
How to use these test files:
fio write.fio
When performance testing ZFS storage pools, we have to work around the ARC (advanced replacement cache) - a RAM based caching mechanism built into ZFS, so our performance results are not skewed. This means we're testing the performance of the underlying storage, and not the speed of our RAM, but its important to note that real world performance may be BETTER than FIO reports because of this.
When testing writes, we can prevent ARC skewing results by forcing writes to be syncronous. This is done for you, as I added "sync=1" in all of the write test files. An alternate method is to set the ZFS dataset to force sync.
Set the stuff dataset in the array zpool to write every file syncronously.
zfs set array/stuff sync=always
Set that dataset back to normal when done testing writes.
zfs set array/stuff sync=standard
When testing reads, the best way I have found to prevent ARC skewing results, is to set the ARC to "metadata" only mode. This means the ARC will only be used for metadata (like where files are stored on the array), and not individual files.
Set the stuff dataset in the array zpool to use ARC only for metadata.
zfs set array/stuff primarycache=metadata
Set that dataset back to normal when done testing reads.
zfs set array/stuff primarycache=all