Veeam Backup and Replication – How to Choose Best Transport Mode for vSphere Proxy?

Davoud Teimouri

2 years ago

I’m sure that most IT administrators are familiar with Veeam’s products and specially, Veeam Backup and Replication (Veeam BR). This product is one of popular backup and replication suite for physical and virtual environments. Veeam Backup and Replication offers three transport modes and we’ll review the transport modes and condition of their selection in this post.

Veeam Backup and Replication Transport Modes for vSphere Proxy

The Veeam Data Mover uses a transport mode to retrieve VM data from the source and write VM data to the target. Job efficiency and time required for job completion greatly depend on the transport mode.

For data retrieval, Veeam Backup & Replication offers the following modes (starting from the most efficient):

Direct storage access
Virtual Appliance (HotAdd)
Network

Network Transport Mode

Network transport mode is the primary transport mode in Veeam Backup and Replication. It’s available for both physical and virtual proxy servers. This mode can impact whole infrastructure because backup data will transfer via hosts network to proxy.

Typically, I’ll go with network mode when other modes are not available and you should consider about some parameters to reduce impact on infrastrcture:

Use network with 10Gb or more bandwidth or use NIC teaming for aggregation.
Put proxy server in same subnet if you have traditional network design in infrastructure.
Use separated management network for backup traffic.

Direct Storage Access

In the Direct storage access mode, Veeam Backup & Replication reads/writes data directly from/to the storage system where VM data or backups are located. This mode comprises two transport modes:

Direct SAN access
Direct NFS access

Direct SAN Access

I recommend using the Direct SAN access transport mode for VMs that have disks located on shared VMFS SAN LUNs connected to ESXi hosts over FC, FCoE, iSCSI, and shared SAS storage.

In the Direct SAN access transport mode, Veeam Backup & Replication leverages VMware VADP to transport VM data directly from and to FC, FCoE and iSCSI storage over the SAN. VM data travels over the SAN, bypassing ESXi hosts and the LAN. The Direct SAN access transport method provides the fastest data transfer speed and produces no load on the production network.

You can use the Direct SAN access transport mode for all operations where the VMware backup proxy is engaged:

Backup
Replication
VM copy
Quick migration
Entire VM restore
VM disk restore
Replica failback

Requirements for the Direct SAN Access Mode

To use the Direct SAN access transport mode, make sure that the following requirements are met:

It is strongly recommended that you assign the role of a VMware backup proxy working in the Direct SAN access mode to a physical machine. If you assign this role to a VM, the VMware backup proxy performance may not be optimal.
A VMware backup proxy using the Direct SAN access transport mode must have a direct access to the production storage using a hardware or software HBA. If a direct SAN connection is not configured or not available when a job or task starts, the job or task will fail.
SAN storage volumes presented as VMware datastores must be exposed to the OS of the VMware backup proxy that works in the Direct SAN access transport mode.

The volumes should be visible in Disk Management but should not be initialized by the OS. If they are initialized, the VMFS filesystem will be overwritten with NTFS, resulting in volumes becoming unrecognizable by ESXi hosts. To prevent volumes initialization, Veeam Backup & Replication automatically sets the SAN Policy within each proxy to Offline Shared and also sets the SAN LUNs to the Offline state.

[For restore operations] A VMware backup proxy must have write access to LUNs where VM disks are located.

Limitations for the Direct SAN Access Mode

The Direct SAN access transport mode can be used to restore only thick VM disks.

The transport mode is applied to the entire VM and not individual virtual disks. This means that each VM can only be processed in one transport mode. If there are any VM disks that cannot be processed in the Direct SAN access transport mode, then the remaining disks of that VM cannot be processed in this transport mode either.

You cannot use the Direct SAN access mode in the following cases:

For VMs residing on vSAN. You can use Virtual appliance and Network transport modes to process such VMs. For details on vSAN restrictions, see VDDK release notes. For example, release notes for VDDK 7.0.3.
If at least one VM disk is located on a VVol.
For Veeam Cloud Connect Replication because in this scenario Veeam Backup & Replication always creates VM replicas with thin disks.
For incremental restore due to VMware limitations. Either disable CBT for VM virtual disks for the duration of the restore process or select another transport mode for incremental restore.
For backing up VM templates.
Veeam Backup & Replication uses the Direct SAN access transport mode to read and write VM data only during the first session of the replication job. During subsequent replication job sessions, Veeam Backup & Replication will use the Virtual appliance or Network transport mode on the target side. The source side proxy will keep reading VM data from the source datastore in the Direct SAN access transport mode.

Veeam Backup & Replication will only write VM data to the target datastore in the Direct SAN access transport mode if the disks of a VM replica are thick-provisioned. However, if the disks are thin-provisioned, Veeam Backup & Replication will write VM data in the Network or Virtual Appliance mode. By default, Veeam Backup & Replication replicates VM disks in the thin format.

To write VM data to the target datastore in the Direct SAN access transport mode, select to convert VM disks to the thick format at the Destination step of the replication job wizard.

IDE and SATA disks can be processed in the Direct SAN access transport mode.

Direct NFS Access

The VMware backup proxy can use the Direct NFS access mode for all operations:

Backup
Replication
Quick migration
VM copy
Entire VM restore
VM disk restore
Replica failback

Requirements for the Direct NFS Access Mode

Direct NFS access mode can be used in VMware vSphere environments running NFS version 3 and 4.1.
The VMware backup proxy used for VM data processing must have access to the NFS datastores where VM disks are located. For more information, see VMware Backup Proxy for Direct NFS Access Mode.
If NFS volumes are mounted on the ESXi host under names, not IP addresses, the volume names must be resolved by DNS from the VMware backup proxy.

Limitations for Direct NFS Access Mode

Veeam Backup & Replication cannot parse delta disks in the Direct NFS access mode. For this reason, the Direct NFS access mode has the following limitations:

You cannot use the Direct NFS access mode for VMs that have at least one snapshot.
Veeam Backup & Replication uses the Direct NFS transport mode to read and write VM data only during the first session of the replication job. During subsequent replication job sessions, the VM replica will already have one or more snapshots. For this reason, Veeam Backup & Replication will use another transport mode to write VM data to the datastore on the target side. The source side proxy will keep reading VM data from the source datastore in the Direct NFS transport mode.
If you enable the Enable VMware tools quiescence option in the job settings, Veeam Backup & Replication will not use the Direct NFS transport mode to process running Microsoft Windows VMs that have VMware Tools installed. We do not use the Direct NFS transport mode because during VM quiescence, VMware creates a snapshot with two delta disks per virtual disk.
If a VM has some disks that cannot be processed in the Direct NFS access mode, Veeam Backup & Replication will process these VM disks in the Network transport mode.

Virtual Appliance (HotAdd) Transport Mode

The Virtual appliance mode is not so efficient as the Direct storage access mode but provides better performance than the Network mode. For VMs assigned as VMware backup proxies, it is recommended to utilize the Virtual Appliance mode.

In the Virtual appliance mode, Veeam Backup & Replication uses the VMware SCSI HotAdd capability that allows attaching devices to a VM while the VM is running. During backup, replication, or restore, the VMware backup proxy attaches the disks of the processed VM. Instead of going through the network, the proxy retrieves or writes VM data directly from/to the datastore.

You can use the Virtual Appliance transport mode for all operations involving the VMware backup proxy:

Backup
Replication
VM copy
Quick migration
Entire VM restore
VM disk restore
Replica failback

Requirements for the Virtual Appliance mode

Ensure that you meet the following requirements to use the Virtual Appliance transport mode:

You must assign the role of a VMware backup proxy to a VM.
The VMware backup proxy and processed VMs must reside in the same datacenter.
The VMware backup proxy must have access to disks of the VM that this proxy processes. For example, in a replication job, the source VMware backup proxy must have access to the disks of the source VM, the target proxy — to the disks of the replica. If a VMware backup proxy acts as both source and target proxy, it must have access to the disks of the source VM and replica. In restore operations, the VMware backup proxy must have access to disks of the restored VMs.
[For NFS 3.0] If you plan to process VMs that store disks on the NFS datastore, you must configure Veeam Backup & Replication to use the proxy on the same host as VMs. This is required due to an issue described in this VMware KB article. For more information on how to configure the proxy, see this Veeam KB article.

As an alternative, you can use ESXi 6.0 or higher and NFS 4.1.

The VMware backup proxy must have the latest version of VMware Tools installed. Note that the backup server installed on a VM can also perform the role of the VMware backup proxy that uses Virtual appliance transport mode. In this case, make sure the backup server has the latest version of VMware Tools installed.
SCSI 0:X controller must be present on a VMware backup proxy. In the opposite case, VM data processing in the Virtual appliance transport mode will fail.

Limitations for the Virtual Appliance mode

To be able to mount the largest virtual disk of hot-added VMs, a VMware backup proxy processing a source VM must reside on a VMFS 3 datastore formatted with the proper block size:

1 MB block size — 256 GB maximum file size
2 MB block size — 512 GB maximum file size
4 MB block size — 1024 GB maximum file size
8 MB block size — 2048 GB maximum file size

This limitation does not apply to VMFS-5 volumes that always have 1 MB file block size.

For vSphere 5.5 and later the maximum supported VMDK size is 62 TB.
[For Microsoft Windows proxy] Prior to executing a data protection task, Veeam Backup & Replication disables the volume automount feature, which remains disabled even after the completion of the data protection task.
Backup and restore of IDE disks in the Virtual appliance mode is not supported.
Backup and restore of SATA disks in the Virtual appliance mode is supported if you use VMware vSphere 6.0 and later.
[For quick migration during Instant Recovery] You cannot use the Virtual Appliance (HotAdd) transport mode if you assign the role of the backup proxy and mount server, or if the backup repository where the backup file is stored, to the same VM.

Conclusion

Choosing Veeam Backup and Replication (Veeam BR) is depends to infrastructure design and backup infrastructure requirements. There are three transport modes, and it is important to consider the technical limitations associated with each mode.

I’ve always recommend using Windows Server as proxy server on physical server with Direct Storage Access transport mode but configuring proxy server network with more than one NIC for failover mode and option to restoring thin disks.

Direct Storage Access needs more resources and also some specific considerations compare to other transport mode, but this mode has lowest impact on infrastructure.

I choose Direct Storage Access always and other modes if it’s no available to us.