Hi everydoby, it's passed a while since I last wrote an article about ZFS;
since I'm handling with it everyday I'll dispatch some trick and some consideration periodically.
For instance, you should keep in mind that with a RAID-Z2 pool you should be aware of the big dataset delete phenomena. If you have a big dataset, let's say 10TB of non-dedupped data with 6 disks RAID-Z2 configuration, and then you delete all the snapshots and the dataset, ZFS will begin the deletion but the operation will last for hours or days, varying obviously with the disk speed and system load.
So, if you have multiple datasets it could be very perilrous launching this kind of operations, because you should keep in mind that the scrub could be launched at any moment, also the SMART long test could appear to be launched. So your system shall very likely crash, throwing offline the zpool.
The bad news? You have to reboot the system.
What's the problem with that you say?
First, you could lost data and corrupt NTFS and BTRFS filesystems. I hope you don't put SQL Server databases in a RAID-Z2 pool, so neither I think about it (doh-doh, prepare to restore the last snapshot!). In some cases event EXT3 filesystems could be corrupted.
But this is the beginning only.
When the operating system (let's clearly say FreeNAS) boots up and tries to imports the ZFS Volume, you'll get the big surprise. Yes, you can't do nothing but wait for long, loong time. We at the FreeNAS forums say it's a 6 hour average, but it could take also a week, or a month.
Why does it take so long?
Because the ZFS transaction log keeps in memory that it has to finish the delete operation. In ZFS terms, it means the system has to find every block for every disk and mark it as free. If you have a 90 disk configuration take an entire holiday to relax. For a month.
So please, please use RAID1 dedicated FreeNAS Server for critical applications and another one for the backups. Or be prepared to wait.