NTFS (New Technology File System) has been the primary file system since Windows XP, as well as Windows-based servers since Windows NT 3.1. And it is showing its age. With the amount of data being processed today, NTFS-based storage devices face serious limitations that need to be overcome. Enter ReFS (Resilient File System), a new file system introduced by Microsoft in 2012.
Among the limitations addressed by ReFS are data and metadata integrity; file, directory, and volume sizes limitations; and resiliency to data corruption with built-in salvage operations that maintain maximum volume accessibility. These changes are not only for large corporations and data centers. They will also benefit many other types of users. Video editors, photographers, audio editors, and file/data hoarders are all likely to find ReFS very useful.
If you’re wondering which improvements ReFS offers in detail, which features are still missing, and how you can use this complimentary file system right now, read on!
What Has Changed from NTFS to ReFS?
NTFS has a number of features that have been removed for ReFS. It only supports a subset of the Win32 file system APIs (Application Programming Interfaces). Compression, file-level encryption via the Encrypting File System (EFS), transactions, hard links, extended attributes, and disk quotas have all been removed. It cannot be used as a boot file system, cannot be used on removable devices, and there is no provision to do an in-place conversion from any other format to ReFS, like there was moving from FAT16 to FAT32 or NTFS file systems.
With all of those features going away, you may wonder what it brings to the table. Let’s look at some of the benefits of using ReFS:
- It uses B+ trees for all on-disk structures, not just metadata, allowing very large files and directories without the performance impact seen with NTFS.
- It use of 64-bit integers for the limits on file size, number of files per folder, total volume size, and number of folders in a volume dwarfs the capabilities of NTFS. The maximum file size is 16 exabytes, maximum number of directories is 18.4 quintillion (short scale), and maximum volume size is 1 yottabyte or 1 trillion terabyes.
- It provides increased resilience by using allocation-on-write for updates, so it is not overwriting existing data during an update.
- It uses large I/O batches to increase performance.
- All metadata has independently stored 64-bit checksums.
- It allows optional integrity streams for file data checksums.
- It eliminates need for CHKDSK or other error-checking tools.
- It maintains compatibility with most existing file system filters.
- It supports BitLocker encryption, ACLs, USN Journaling, change notifications, symbolic links, junction points, mount points, reparse points, volume snapshots, file IDs, and opportunistic locking.
- It leverages existing file system APIs.
- Metadata is periodically scrubbed by reading it and performing checksum verification.
Storage Spaces in Windows 8 & Up
When ReFS is used in conjunction with mirrored storage using Storage Spaces, a storage virtualization layer built into Windows 8 and later, it can also perform automated error correction. The same type of data scrubbing that is performed on the metadata can be optionally enabled for data files on mirrored drives. In this case, all mirrored copies of files are read and verified using their checksums. Any bad copies found will be replaced with good copies of the file.
What Is the Development Status of ReFS?
Users do need to be mindful that ReFS in its current iteration is not meant to be a replacement for NTFS. Instead, it is a complimentary file system, designed to handle tasks where NTFS falls short, such as file and data archival servers.
Later versions of ReFS may very well replace NTFS as the default Windows file system, but it isn’t going to happen soon. After all, it took NTFS 8 years from its introduction until it became the default file system for consumers in Windows XP.
Despite some rumors, this is not a beta product, although it is not yet feature complete. Much like the development of NTFS, it will continue to gain features during its lifetime. Some of the features it is “missing” compared to NTFS will come back. Disk quotas, encryption, and the ability to use it as a boot device are the immediate candidates that come to mind.
How to Use ReFS Now?
There are two different ways you can use ReFS on Windows 8.1 and 10. The first is by creating a mirrored storage space and formatting that with ReFS. The second involves a temporary registry hack that will allow you to format a single non-mirrored drive with ReFS.
How to Implement ReFS With Storage Spaces
To use ReFS in conjunction with storage spaces, you will need at least two currently unused physical hard drives or you can create virtual hard drives instead.
To create VHD files, open the Disk Management console (right-click Start > Disk Management) then select Action > Create VHD. Supply a name for the file, choose a size, and ensure that the Fixed size option is selected. Using a thin-provisioned (dynamically sized) virtual disk may cause problems if you run out of physical hard drive space before the virtual disk has reached its maximum size. Click on OK to create and attach the VHD to the system.
Once you have both hard drives ready for use, you need to create your storage pool and the new virtual (mirrored) drive.
Open Control Panel > Storage Spaces and click the link to Create new storage pool and storage space.
Select the unused drives you want to add to the storage pool and click Create pool.
Next, give your storage space a name (effectively a volume label), drive letter, and select the REFS format for the new virtual drive. In order to use ReFS, the Resiliency type must be either Two-way mirror or Three-way mirror. Selecting Simple or Parity will revert to NTFS. The note under the drop-down list will tell you the minimum number of drives required to use that option. Click Create storage space when done.
This will create, format, and mount the drive, which is then ready for use.
How to Implement ReFS Without Storage Spaces
There is a way to use a single, non-mirrored drive as an ReFS drive, but it does lose some of its resiliency advantages when used in this manner. Keep in mind that a drive formatted as ReFS cannot be used as a boot device.
To do this, you will need to create a new registry key. Open the registry editor (Start > type regedt32 > Enter). Navigate to the key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control. Create a new key (right-click on the Control leaf > New – Key) called MiniNT. Under MiniNT, create a new DWORD (right-click on empty space in the right pane > New – DWORD) called AllowRefsFormatOverNonmirrorVolume and give it a value of 1. Reboot your computer for this change to take effect.
Once you have rebooted your computer, you can now format a drive using the GUI, or the command line tools format or diskpart. Using the GUI to do this will leave integrity streams disabled since it is not using mirrored drives. Using the format command from the command line, you will need to add the /fs:refs flag to format it as an ReFS drive and integrity streams for data may be enabled by using the /i:enable option.
Once the drive has been formatted, open the registry editor again and remove the MiniNT key we previously added, then reboot your computer. Leaving this key in may cause problems on your computer by making some components think it is running in a preinstallation environment.
How Well Does It Work?
I have been playing with this off and on for a little over a month and have not run into any problems at all. I have even disconnected drives and shut down the system while copying files to see if it lives up to its name. Overall, I must say “resilient” really does describe this file system. Even with the abuse I have handed out, I have yet to lose a single byte of data.
Even so, this does not eliminate the need for good backups. A meteor crashing through your computer isn’t likely to leave enough hardware behind for any kind of recovery.
I was planning on building a FreeNAS server for my home network next year. This has me considering building a Windows 2016 storage server instead.
Has anyone else been dabbling with ReFS or are you now considering a look at it? Have you had any issues with it? Were you able to work around them? Let us know in the comments below!
Image credit: File cabinets by Andrea Danti via Shutterstock.com