Agent for Virtual Machines

The following issues relating to the Agent for Virtual Machines are known to exist in this release:

For VMware:

• The Agent does not support recovering virtual machines to data stores that are named using the special characters asterisk, the at symbol, and the German umlaut character. Backup jobs for virtual machines named using these characters may fail.
• When VMware vSphere High Availability (HA) or VMware vMotion is in progress during a backup job, the backup job fails. Once VMware vSphere HA or VMware vMotion completes, run the backup job again and it will complete successfully.
• After a successful recovery of VMware virtual machines, the Disk Provisioning Type of the Virtual Hard Disk depends on the backup approach that you use.
  • When you perform thin disks backup using VDDK, and the backup is not an active block backup, you can recover the disk as a thick disk.
  • When you perform thin disks backup using VDDK, and the backup is an active block backup, then you can only recover that disk as a thin disk.

• Backup jobs associated with Windows Server 2008 R2 and Windows 7 VMs running on VMware ESX Server systems may fail when VMware attempts to create snapshots of the VM.
• For more information, refer to the Knowledge Base article on the VMware website:
• Using VDDK approach, the Agent for virtual machines do not support backup of a virtual machine which itself is a backup proxy Server. To backup this virtual machine, you must use a different backup proxy Server.

• By default, VMware VDDK 7.0 U2 is packaged with the Arcserve Backup 19.0. VDDK 7.0 U2 does not support ESX Server 6.0 and vCenter Server 6.0, therefore backing up or restoring virtual machines on these Servers may fail. To resolve this issue, you can download and install VDDK 5.5 and change the VDDKDirectory registry key to the location where VDDK 7.0 U2 is installed.
• When VMware vSphere High Availability (HA) or VMware vMotion is in progress during a backup job, the backup job fails. Once VMware vSphere HA or VMware vMotion completes, run the backup job again and it will complete successfully.
• When VMware-based virtual machines contain independent disks and raw device mappings that are configured in the physical mode, Arcserve skips these volumes during backup jobs. However, the fact that the volumes were skipped during backup jobs is not recorded in the Activity Log.
• When the process of retrieving the used data blocks on VMware-based virtual machines using VMware changed block tracking (CBT) fails, the Agent currently cannot reset CBT when the virtual machine contains snapshots. To correct this behavior, remove the snapshot manually so that you can reset CBT. VMware is aware of this problem and is working to provide a solution to this behavior.
• Restoring C:\ volume from raw session to the original location fails for Windows 2012 and later versions. Use the Recover VM option to restore.
• VM Agent installation is not supported on x86 Servers with the Arcserve Backup 19.0. However, if you choose any other agent and perform a remote installation or create a response file, the local installation option is enabled. Close the setup windows and try again.
• Symptom
• VMware uEFI VM with Windows 2019 backup failure, the error message displayed is Creation of the virtual machine snapshot has failed. The issue occurs when disk.EnableUUID is set to True.
• Solution
• 1. Follow the steps to verify the value of disk.EnableUUID:
     1. Right-click the virtual machine and click Edit settings.
     2. Click the VM Options tab.
     3. Navigate to Advanced > Configuration Parameters > Edit Configuration.
     4. Search for disk.EnableUUID parameter and verify the value.
  2. 
  2. Follow the steps to change the value to FALSE if the value of disk.EnableUUID to TRUE
     1. Modify the value of disk.EnableUUID to FALSE.
     2. Click OK to save.
     3. Click OK to exit.
     4. Reboot the virtual machine for changes to reflect.
     5. Note: After changing disk.EnableUUID to FALSE, please be aware that VM backup will be successful with Crash-consistency only. We recommend customers to use Arcserve Backup application agents to protect applications running on this virtual machine.

  What is disk.EnableUUID parameter used for?

  disk.EnableUUID is a parameter for VMWare virtual machine, which enables the snapshots to be application-consistent or crash-consistent.

  Application-Consistent snapshots means that the snapshot has everything that was in memory/RAM at the time of the snapshot. All the running programs (Applications), processes, and even windows that were open at the time of the snapshot can be restored.

  Crash--Consistent is just the opposite. It doesn’t have everything that was in memory at the time of the snapshot. On a restore, the VM acts as if it is being started up after a Crash. All files are intact but everything that was in memory is lost.

  Note: VMs created on 4.1 or later have this parameter enabled by default. We recommend not to set disk.EnableUUID=FALSE on VMs running applications except in this case till VMware fixes this issue.

For Hyper-V:

• On Windows Server 2008 systems with the Hyper-V role enabled, performing a full backup of Hyper-V VSS writer may sometimes cause the operating system to generate warning messages about the removal of the storage miniport driver, such as:
• Device 'Storage miniport driver' VMBUS\{d6b9a233-a170-4602-9863-78f7c287c7df}\1&3189fc23&0&{d6b9a233-a170-4602-9863-78f7c287c7df}) has been lost from the system because the preparation for removal was not performed first.
• Device 'Msft Virtual Disk SCSI Disk Device'(SCSI\Disk&Ven_Msft&Prod_Virtual_Disk\2&7aef4bc&0&000000) been lost from the system because the preparation for removal was not performed first.
• Due to a known issue with Microsoft Hyper-V VSS writer, the warning messages are generated and it does not affect the success of backup jobs. You can still successfully perform data restore jobs. When all virtual machines under Microsoft Hyper-V VSS Writer are displayed as Backup Using Saved State-virtual machine name, this problem does not occur.
• On Windows Server 2008 systems with the Hyper-V role enabled, the VM backup job may fail in certain environments, and records the following errors in the Activity Log:
• E12586 This VSS Writer (\\TX200S3x64\Microsoft Hyper-V VSS Writer) has reported a failure during the freeze event.
• E12606 Selected Writer option cannot be obtained.
• E12586 This VSS Writer (\\TX200S3x64\Microsoft Hyper-V VSS Writer) has reported a failure during the freeze event.
• Due to a known issue with the Microsoft Hyper-V VSS writer, the Hyper-V VSS writer crashes.
• When you restore from an internal session of a nameless volume to an original location, it fails with an invalid drive specification (error code AE0060).
• When multiple Hyper-V virtual machines contain multiple disks that are sharing a common parent disk; the process of recovering any of the machines in a running state can fail. This behavior is expected when the process of recovering Hyper-V virtual machines tries to overwrite the parent disk where another virtual machine is using it. Under this condition, the job appears to complete successfully, however, the Hyper-V virtual machine is not going to start.
• When you perform a virtual machine backup using agent and the virtual machine name on the Hyper-V manager is more than 64 characters, the backup job fails.
• When a Hyper-V virtual machine contains data (VHD/configuration files) residing on remote SMB shares, verify that the remote share is present before you start the VM recovery job to the original location.
• Note: You cannot access application-based writers when performing disaster recoveries. The solution to this behavior is to recover the system state and the volumes, bring the computer online, and then recover the writers. You can then access the application data on remote shares after the application writer recovery starts.
• When you perform backup on VMware or Hyper-V virtual machine with application-based writer data that resides on SMB shares, the backup job fails.
• The Agent does not support restoring the data from the VHDX transaction log, when recovering virtual machines. The Agent behaves in this manner because the Hyper-V VSS writer snapshot does not quiesce or commit the transaction log data. In addition, the Hyper-V VSS does not provide a provision in the VSS snapshot API.
• Hyper-V VSS Writer backup jobs can fail when using the Agent to backup virtual machines running on Windows Server 2012 operating systems with the iSCSI Target Storage Provider (VDS and VSS hardware providers) feature enabled. The following message appears in the Activity Log:
• The volume shadow service provider had an unexpected error. The error code is logged in the error log.
• To correct this behavior, configure the iSCSI storage provider identity credentials on the Hyper-V Server. For more information, see Step 4 of the following article in the Microsoft Windows Server TechNet Library:
• http://technet.microsoft.com/en-us/library/hh848268
• RAW backup with file level restore is not supported for ReFS file systems. Therefore, file level restore is not available with RAW backup.
• Due to a Microsoft known issue, when you configure storage space on a VMware or Hyper-V virtual machine, backing up the virtual machine fails with a snapshot creation error.
• The version of Windows Hyper-V that is running on the target Server is older than the version of Windows Hyper-V that is running on the Server that was backed up. To resolve this issue, restore the virtual machine that was backed up on the Windows Hyper-V Server to an equal or later version of a Windows Hyper-V Server.
• The Agent does not support backing up Hyper-V virtual machines that contain disks that are equal to 64 terabytes.
• When recovering data from NTFS Deduplication volumes that are optimized and a volume contains a very large quantity of files (1 million or more files), the throughput calculation in the Activity Log and the Job Monitor is incorrect.
• Restoring C:\ volume from rawif session to the original location fails for Windows 2012 and 2012 R2. Use Recover VM to restore.
• VMAgent installation is not supported on x86 Servers with the Arcserve Backup 19.0. However, if you choose any other agent and perform a remote installation or create a response file, the local installation option is enabled. Close the setup windows and try again.

• When you backup a virtual machine using the Agent and the virtual machine name on the Hyper-V manager is more than 64 characters, the backup job fails.
• When a Hyper-V virtual machine contains data (VHD/configuration files) residing on remote SMB shares, verify that the remote share is present before you start the VM recovery job to the original location.
• Note: You cannot access application-based writers when performing disaster recoveries. The solution to this behavior is to recover the system state and the volumes, bring the computer online, and then recover the writers. You can then access the application data on remote shares after the application writer recovery starts.
• When you backup a VMware or Hyper-V virtual machine with application-based writer data that resides on SMB shares, the backup job fails.
• The Agent does not support restoring the data from the VHDX transaction log, when recovering virtual machines. The Agent behaves in this manner because the Hyper-V VSS writer snapshot does not quiesce or commit the transaction log data. In addition, the Hyper-V VSS does not provide a provision in the VSS snapshot API.
• Hyper-V VSS Writer backup jobs can fail when using the Agent to backup virtual machines running on Windows Server 2012 operating systems with the iSCSI Target Storage Provider (VDS and VSS hardware providers) feature enabled. The following message appears in the Activity Log:
• The volume shadow service provider had an unexpected error. The error code is logged in the error log.
• To correct this behavior, configure the iSCSI storage provider identity credentials on the Hyper-V Server. For more information, see Step 4 of the following article in the Microsoft Windows Server TechNet Library:
• http://technet.microsoft.com/en-us/library/hh848268
• RAW backup with file level restore is not supported for ReFS file systems; therefore file level restore is not available with RAW backup.
• Due to a Microsoft known issue, when you configure storage space on a VMware or Hyper-V virtual machine, backing up the virtual machine fails with a snapshot creation error.
• IP address is not displayed in the properties of Windows Server 2016 VM on Microsoft Hyper-V.
• The version of Windows Hyper-V that is running on the target Server is older than the version of Windows Hyper-V that is running on the Server that was backed up. To resolve this issue, restore the virtual machine that was backed up on the Windows Hyper-V Server to an equal or later version of a Windows Hyper-V Server.
• The Agent does not support backing up Hyper-V virtual machines that contain disks that are equal to 64 terabytes.
• When recovering data from NTFS Deduplication volumes that are optimized and a volume contains a very large quantity of files (1 million or more files), the throughput calculation in the Activity Log and the Job Monitor is incorrect.
• Symptom
• In Hyper-V 2016, for any VM that is backed up and restored to its original location, the restore is successful. However, the VM is restored into Saved State. The issue occurs due to Hyper-V Integration component that is installed on the guest operating system is getting incompatible with the target Hyper-V Server.
• Solution
• From Windows 2016, integration services are available through windows update and has no GUI enabled function. You need to upgrade integration services and run the windows update on both Hyper-V host servers and VM’s.
• If the VM is in power off state, a check point is created under the check points section. You need to right-click on the checkpoint, select revert, and turn on the VM.
• Symptom
• In Hyper-V 2016, for any VM that is backed up and restored to its original location, the restore is successful and is saved state. The issue occurs due to Hyper-V Integration component that is installed on the guest operating system getting incompatible with the target Hyper-V Server.
• Solution
• From Windows 2016, integration services are available through windows update and has no GUI enabled function. You need to upgrade integration services and run the windows update on both Hyper-V host servers and VM’s.
• If the integration services are up to date and VM is restored in saved state, follow these steps:
1. Right click on the VM.
2. Select the option Delete saved state.

VM turns off.

3. start the VM.
• The disks under the migrated VM may not appear.
• Symptom
• Due to Hyper-V limitation, when you perform live migration of VMs from one Hyper-V host to another Hyper-V host server (any version of Hyper-V), the disks are not listed under migrated VM. The issue occurs because the migration of VM fails to get the Arcserve Server details from CAW Agent admin window.
• Solution
• As a workaround, follow these steps:
1. Log into the migrated VM.
2. Navigate to Backup Agent Admin, Client Agent configuration.
3. Navigate to Agent for VM Tab.
4. Add Arcserve Backup Server details and click on OK.
5. Stop and Start the Agent Services.