From the Lab
VhdTool Is Dead, Long Live VhdxTool!
Systola introduces a successor to VhdTool for quickly creating and expanding virtual disks and discusses security aspects of using this tool.
Roman Kuznetsov @ 29.10.2013
Follow Roman Kuznetsov on LinkedIn
With VHDX, Microsoft introduced a new format for virtual data media some time ago. Previously, the comparatively simpler VHD format was used for a wide variety of systems.
VHD was used for file servers, some iSCSI target servers, but above all, of course, for Hyper-V servers. It was used and abused for a variety of purposes; for example, it was used as a generic container, and some backup systems even support complete backups of VHD containers and the subsequent restoration of individual elements.
Note: There are two types of virtual disks: fixed-size disks and dynamically growing virtual disks. Since this article is about production systems, only fixed-size disks are discussed.
Managing VHD files was a problem that should not be underestimated. On Windows XP or Server 2003, it was anything but easy to create and manage VHD files unless you had an MS iSCSI target at hand. Windows 7 / Server 2008/R2 partly solved the problem by integrating the management of VHD files into WMI and also into the GUI. This made it possible to create and expand VHD volumes via Powershell, the MMC Disk Management snap-in or Diskpart.
But the main problem remained: if you wanted to create a virtual disk of decent size, say a few hundred gigabytes, you had to spend a lot of time. Even on fast hardware, creating such disks took forever, not to mention slow computers or hard disks. The reason for this was a potential security risk: to prevent old data from being recovered from the newly allocated VHDs, the entire payload of the new volume was zeroed out. Details can be found on Ben Armstrong's blog (link is now dead).
Microsoft's solution: VhdTool
More than half a year later, the guys from Microsoft finally brought out a tool that did without zeroing out the new data carrier and could therefore also create VHDs in gigabyte and terabyte sizes in no time at all. Incidentally, the same functionality was also available in the MS iSCSI Target Server in Storage Server 2003 R2. However, if I remember correctly, it was removed again with Storage Server 2008. Further information could earlier be found in the announcement of the tool (link dead again).
More than half a year later, the guys from Microsoft finally brought out a tool that did without zeroing out the new data carrier and could therefore also create VHDs in gigabyte and terabyte sizes in no time at all. Incidentally, the same functionality was also available in the MS iSCSI Target Server in Storage Server 2003 R2. However, if I remember correctly, it was removed again with Storage Server 2008. Further information can be found in the announcement of the tool.
We will discuss in a moment what has to be considered when using such tools in production. But first it is of interest that with the introduction of Server 2012 and the new Hyper-V version, the old VHD format has been abandoned and replaced by the new and more reliable VHDX.
The new format is very practical and has numerous advantages. However, since its introduction, Hyper-V users have again suffered from sluggish processing when creating or resizing virtual disks. The internet forums are overflowing with questions about how to speed up the creation of VHDX files. The reader already suspects it: the already mentioned VhdTool does not support the new VHDX format, as it is fundamentally different from VHD.
For Systola as a provider of DaaS-Solutions, working with virtual data carriers is part of our daily bread. Our team was therefore also faced with this problem. After failing to find a successor for VhdTool that could cope with the new format, we decided to write the much-needed tool ourselves.
Systola introduces: VhdxTool
After a thorough study of the specifications, we now have a fully functional utility that can save you a lot of time. But before we take a closer look at the new programme, we first need to talk about the problem caused by its use and determine when the programme can be used and when it is better not to.
There is a fundamental problem with reusing "used" space on the underlying mass storage for a new virtual disk: if the space reserved for the virtual disk is not zeroed out, the free storage areas of the virtual disk created will most likely contain valid or partially valid data. This data can easily be recovered by the virtual machine to which the virtual disk is assigned. This must be prevented.
Better safe than sorry
It is not particularly difficult to distinguish the "dos and don'ts" when using this tool to create VHDX data carriers. In the following, "end users" means either your colleagues or your customers or clients, depending on the type of company you work for.
DO use this tool:
DON’T use the tool:
Once the scenario is clear to you and you find yourself on the DO-side, download a copy of VhdxTool, make sure your system has .NET 4.5 or greater installed, fire up an elevated command prompt or powershell console and let’s start VHDXing!
There are five operations of the tool at the time of writing. Therefore four main modes of execution and one extra informational mode:
vhdxtool.exe create [options] - create a new VHDX
vhdxtool.exe extend [options] - extend an existing VHDX
vhdxtool.exe convert [options] - convert a dynamic VHDX to a static one
vhdxtool.exe upgrade [options] - convert a VHD to VHDX
vhdxtool.exe display [options] - view information about an existing VHDX
The options are as follows (use either short or long form):
VHD was used for file servers, some iSCSI target servers, but above all, of course, for Hyper-V servers. It was used and abused for a variety of purposes; for example, it was used as a generic container, and some backup systems even support complete backups of VHD containers and the subsequent restoration of individual elements.
Note: There are two types of virtual disks: fixed-size disks and dynamically growing virtual disks. Since this article is about production systems, only fixed-size disks are discussed.
Managing VHD files was a problem that should not be underestimated. On Windows XP or Server 2003, it was anything but easy to create and manage VHD files unless you had an MS iSCSI target at hand. Windows 7 / Server 2008/R2 partly solved the problem by integrating the management of VHD files into WMI and also into the GUI. This made it possible to create and expand VHD volumes via Powershell, the MMC Disk Management snap-in or Diskpart.
But the main problem remained: if you wanted to create a virtual disk of decent size, say a few hundred gigabytes, you had to spend a lot of time. Even on fast hardware, creating such disks took forever, not to mention slow computers or hard disks. The reason for this was a potential security risk: to prevent old data from being recovered from the newly allocated VHDs, the entire payload of the new volume was zeroed out. Details can be found on Ben Armstrong's blog (link is now dead).
Microsoft's solution: VhdTool
More than half a year later, the guys from Microsoft finally brought out a tool that did without zeroing out the new data carrier and could therefore also create VHDs in gigabyte and terabyte sizes in no time at all. Incidentally, the same functionality was also available in the MS iSCSI Target Server in Storage Server 2003 R2. However, if I remember correctly, it was removed again with Storage Server 2008. Further information could earlier be found in the announcement of the tool (link dead again).
More than half a year later, the guys from Microsoft finally brought out a tool that did without zeroing out the new data carrier and could therefore also create VHDs in gigabyte and terabyte sizes in no time at all. Incidentally, the same functionality was also available in the MS iSCSI Target Server in Storage Server 2003 R2. However, if I remember correctly, it was removed again with Storage Server 2008. Further information can be found in the announcement of the tool.
We will discuss in a moment what has to be considered when using such tools in production. But first it is of interest that with the introduction of Server 2012 and the new Hyper-V version, the old VHD format has been abandoned and replaced by the new and more reliable VHDX.
The new format is very practical and has numerous advantages. However, since its introduction, Hyper-V users have again suffered from sluggish processing when creating or resizing virtual disks. The internet forums are overflowing with questions about how to speed up the creation of VHDX files. The reader already suspects it: the already mentioned VhdTool does not support the new VHDX format, as it is fundamentally different from VHD.
For Systola as a provider of DaaS-Solutions, working with virtual data carriers is part of our daily bread. Our team was therefore also faced with this problem. After failing to find a successor for VhdTool that could cope with the new format, we decided to write the much-needed tool ourselves.
Systola introduces: VhdxTool
After a thorough study of the specifications, we now have a fully functional utility that can save you a lot of time. But before we take a closer look at the new programme, we first need to talk about the problem caused by its use and determine when the programme can be used and when it is better not to.
There is a fundamental problem with reusing "used" space on the underlying mass storage for a new virtual disk: if the space reserved for the virtual disk is not zeroed out, the free storage areas of the virtual disk created will most likely contain valid or partially valid data. This data can easily be recovered by the virtual machine to which the virtual disk is assigned. This must be prevented.
Better safe than sorry
It is not particularly difficult to distinguish the "dos and don'ts" when using this tool to create VHDX data carriers. In the following, "end users" means either your colleagues or your customers or clients, depending on the type of company you work for.
DO use this tool:
- in labs and testing environments without any restrictions
- on the new hardware, meaning new, unused and unwritten underlying HDDs
- on used HDDs if they all (!) were zeroed-out beforehand
- for non-system VHDXs that will not be directly accessible by end-users (including admins) of the system, for example for SQL-Server or Exchange databases
- for any internally used VHDXs that will never be exposed to end-users
DON’T use the tool:
- for system VHDXs of the systems accessible by end-users
- for non-system VHDXs of the systems accessible by end-users
- for VHDXs accessible via SMB or other network protocols
- for VHDXs that might be transferred to an end-user
- for golden images of VHDXs for TS or VDI environments
- for other VHDXs-templates that might end up being used in end-user systems
Once the scenario is clear to you and you find yourself on the DO-side, download a copy of VhdxTool, make sure your system has .NET 4.5 or greater installed, fire up an elevated command prompt or powershell console and let’s start VHDXing!
Command Syntax, The System’s Sin-Tax
There are five operations of the tool at the time of writing. Therefore four main modes of execution and one extra informational mode:
vhdxtool.exe create [options] - create a new VHDX
vhdxtool.exe extend [options] - extend an existing VHDX
vhdxtool.exe convert [options] - convert a dynamic VHDX to a static one
vhdxtool.exe upgrade [options] - convert a VHD to VHDX
vhdxtool.exe display [options] - view information about an existing VHDX
The options are as follows (use either short or long form):
BlockSize, LogicalSectorSize, PhysicalSectorSize are the same parameters you would usually provide to the New-VHD cmdlet. For normal operations you can just leave these parameters out and the default values will be used, these are fine for most cases. If you have newer 4K hard drives, you might need to tweak the sector size, in which case these parameters will be your friends. Also, consult these two resources on sector size choices:
Microsoft support policy for 4K sector hard drives in Windows and
Hyper-V Support for Large Sector Disks Overview.
Examples:
Microsoft support policy for 4K sector hard drives in Windows and
Hyper-V Support for Large Sector Disks Overview.
Examples:
Creates a new VHDX, 800 megabytes big.
vhdxtool.exe create -f test.vhdx -s 800MB
Creates a new 4K-VHDX with the size of 2 terabytes.
vhdxtool.exe create -f test.vhdx -s 2TB -b 64 -l 4096
Extends an existing VHDX to the size of 80 gigabytes.
vhdxtool.exe extend -f test.vhdx -s 80GB
Displays internal information about an existing VHDX.
vhdxtool.exe display -f test.vhdx
There are even more good news for those, who are used to the old syntax of Vhdtool: we’ve made our tool also compatible with vhdtool-syntax, so you can just type:
vhdxtool.exe /create <FilePath> <DiskSize>
vhdxtool.exe /extend <FilePath> <NewSize>
Conclusion
With VhdxTool you can now create virtual hard disks of any size with blazing speed. Whether you want a 10GB or 10TB VHDX - it does not matter, VhdxTool will need the same amount of time for any size, just provide enough space on the storage side.
VhdxTool requires .NET 4.5, but we assume you will run it on WIndows Server 2012/R2 most of the time, where this component should be already installed.
You should always observe the applicability of this tool before going into production and consider all the risks you may face. If in doubt, don’t use the tool and go for a built-in solution, but if you know what you are doing - here is your time saver.
Happy VHDXing!