[Review]: VMware vSphere APIs for I/O Filtering (VAIO)
What’s vSphere APIs for I/O Filtering (VAIO)?
This is not a new features on last vSphere version but I went to write a post about that. We had to deploy different replication scenarios for our customers and some customers needs Point-In-Time Recovery (PiT) and The PiT solutions using VAIO actually so learning about the API is necessary for administrators.
The vSphere APIs for I/O Filtering (VAIO) were introduced in vSphere 6.0 Update 1. The VAIO framework and program were developed to provide VMware and partners the ability to insert ﬁlters for I/O into the data path of virtual machines. These “I/O Filters” enable VMware, and partners, to intercept and manipulate the I/O. This manipulation can provide open-ended data services, but thus far is limited to four use cases, two of which are currently exclusive to VMware and two which are open for partners. These use cases are: Replication, Caching, Quality of service (VMware only), Encryption (VMware only).
Prior to VAIO, partners had to resort to unsupported kernel level methods or ineﬃcient virtual appliances to intercept and manipulate virtual machine I/O streams. This led to a variety of solutions, which were all implemented in diﬀerent ways, and could potentially result in instability and operational complexity. The VAIO framework and program was speciﬁcally designed to address these challenges. Note that the program does entail a vetting and certiﬁcation process for partners before a solution is oﬃcially VAIO certiﬁed. This is to ensure stability and provide a consistent user experience.
How VAIO Works?
The vSphere APIs for I/O Filtering enables the interception of I/O requests from Guest Operating Systems (GOS) to virtual disks (VMDK). This is enabled by inserting an I/O Filter in the vSphere I/O stack. This ﬁlter driver executes in user space (called User Worlds in ESXi) so that 3 rd party ﬁlter code can run natively in ESXi without jeopardizing the stability of the ESXi kernel. At the same time, VAIO allows for ﬁlter drivers to intercept I/O operations as they go through the data path of the ESXi kernel without any perceived performance overhead. I/O operations are intercepted as soon as they go through the virtual SCSI emulation layer (vSCSI) in the kernel.
As a result, data services can be applied before I/O is processed by the diﬀerent storage virtualization modules of ESXi’s kernel (VMFS, NFS, SCSI, vSAN, etc), which allows for generic data services that are agnostic to the storage backend. For the same reason, I/O is intercepted before it traverses the network providing security and integrity of data.
In the example below, a single I/O Filter is shown. The I/O Filter is implemented in the user space/ world to avoid impacting the hypervisor should there be an issue with the I/O Filter. It is also possible to have multiple I/O Filters applied to the same VM or VMDK(s). If multiple I/O Filters are applied to the same VM or VMDK, the order in which it is applied is deﬁned by the ﬁlter “class order” which is deﬁned as part of the VAIO framework. This means, for instance, that a replication ﬁlter executes before a cache ﬁlter. Once the I/O has moved through all I/O ﬁlters, it moves to the next layer.
The I/O Filter not only intercepts and manipulates I/O depending on the implemented functionality, but it can also be responsible for acknowledging the I/O to the Guest Operating System. It is a bidirectional information and traﬃc ﬂow. If an I/O Filter is written to do write caching, this can be implemented in a write-back or write-through modus. Depending on the implementation, the I/O Filter needs to be informed that the write data is committed to a storage device before placing it in cache.
When a replication I/O Filter is implemented, the replication can be done either synchronously or asynchronously. Synchronous replication includes the latency to reach the remote storage in the critical path of a write operation both copies of the operation must be persisted before the operation is acknowledged to the Guest OS. Asynchronous replication is used when the additional latency is not acceptable. This means that the acknowledgement from the local storage system enables the VM to continue, while the I/O is (asynchronously) being replicated to a diﬀerent location. Note that in this case, the successful asynchronous replication of the I/O is still acknowledged to the I/O Filter. The I/O ﬁlter will keep track of each I/O request on a per VMDK basis to ensure data consistency and integrity.
Besides regular I/O, VAIO is also informed when certain control operations are performed. For instance, when a snapshot is requested by someone (or something), or a change to a VM or VMDK is requested (grow disk, stop VM, start VM, etc.) the I/O Filter is informed about this event so that this change can be taken in to account.
Requirements for implementing I/O Filters usually depend on the type of ﬁlter being used. There are, however, a couple of common requirements before an I/O Filter can be installed and conﬁgured:
- vSphere 6.0 Update 1 or higher
- vMotion with attached ﬁlters is supported, but requires the I/O Filter to be installed on the target host
- DRS is supported, but requires the I/O Filter to be installed on all hosts in the cluster
- Note that with vSphere 6.0 U1 it was required to have DRS enabled, as of vSphere 6.0 U2 this is no longer the case.
- Permissions to install a VIB (vSphere Installation Bundle)
You can read more about that on this online white paper: https://storagehub.vmware.com/t/vsphere-storage/vmware-vsphere-apis-for-i-o-filtering-vaio/