Multipath: Best and Safe Solution Map in Use Devices

Based on Red Hat solution (https://access.redhat.com/solutions/47438), the below steps should be done before flush any multipath device, otherwise the command will be failed:

  • Locate any subsystem or process holding the device open.
  • For any subsystem or process holding the device open, stop the process, or issue commands to release the device.
  • Some examples of possible holders of a device and the commands to release it:
  • One or more partition mapping(s) still exists on the device.
    • Use “kpartx -d” on the device to remove the device partition mapping(s).
  • A filesystem exists on the device and is currently mounted.
    • Unmount the filesystem and if it exists in /etc/fstab, remove it.
  • The device was used by LVM, and still has device mapper state in the kernel.
    • Use “lvchange -an” to deactivate any logical volume(s) associated with the device.  A list of logical volumes associated with the device may be found by examining the output of “lvs -o +devices“.
    • If “lvchange -an” fails, the logical volume is only partially removed, or there are blocked processes with I/O outstanding on the device, use “dmsetup remove -f” followed by “dmsetup clear” on the device.  See dmsetup man page for full explanation of these commands.

Once all holders of the device have been removed, the device should be flushed with “multipath -f“.

But may the above instruction failed at first step because the disk is hold by a process.

Multipath

Problem!

Here is an example of the error:

multipath -f {Multipath Alias}
Mar 1 09:59:10 | {Multipath Alias}: map in use
dmsetup remove -f {Multipath Alias}
device-mapper: remove ioctl on {Multipath Alias} failed: Device or resource busy

Solution!

What is the solution for this problem?

As Red Hat mentioned, “Locate any subsystem or process holding the device open”. Usually a process holding the disk or disks, so as the first step follow the below:

lsof | grep {Multipath Alias}

Output will be same as the below:

oracle    10298    oracle  257u      BLK              252,8       0t0 18446612177448267048 /dev/oracleasm/disks/{Multipath Alias} (deleted)

In this example output, oracle process holding the disk and the process should be identified before any action:

ps -ef | grep 10298
oracle   10298     1  0  2016 ?        00:00:00 oraclecmsrestd (LOCAL=NO)

If there is no problem about killing the process, kill the process by the below command:

kill -9 10298

Then try to flush device again and if the problem is still exists, follow the below for troubleshooting:

  • Use ‘lsof’ to attempt to find anyone holding the device open.
  • Check ‘dmsetup’ output for any device mapper maps that depend on the device
  • Check the if there is a device for mpath7 in /dev/mapper/mpath7
  • Check multipath -v4 -ll

Further Reading

HBA Port and Linux: What’s The Best Solutions to Disable?

VMware Horizon View Connection (Session) Duration Report

Davoud Teimouri

Professional blogger, vExpert 2015/2016/2017/2018/2019/2020/2021/2022/2023, vExpert NSX, vExpert PRO, vExpert Security, vExpert EUC, VCA, MCITP. This blog is started with simple posts and now, it has large following readers.

Leave a Reply

Your email address will not be published. Required fields are marked *