Multipath 切换主路径

前言

本文记录multipath相关指令, Multipath的path path_grouping_policy – Paths are grouped into path groups. The policy determines how path groups are formed. There are five different policies.

failover: One path per priority group
multibus: All paths in one priority group. This is the default.
group_by_serial: One priority group per storage controller (serial number)
group_by_prio: One priority group per priority value
group_by_node_name: One priority group per target node name

有时候使用failover，需要切换主路径，本文记录切换主路径的方法

查看当前配置

root@host243:~# multipath -ll
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running

切换主路径

从上面的输出可以看出，sdd/sde/sdf/sdg 都是同一个设备的4条不同的路径。如果设备是分布式存储提供的iSCSI设备，每条路径可能对应一个Gateway。有时候可能需要切换主路径，比如某个存储节点要关机维护，这种情况，要怎么做呢？

首先执行如下语句，进入交互页面：

multipathd -k

通过如下指令可以实时查看某个multipath的多个group信息：

multipathd> show topology
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running
multipathd>

对于我这个环境比较简单，如果存在多个，可以通过如下指令指定：

multipathd> show map 2c2c5d201aba719ee topology
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running
multipathd>

从上面的输出可以看出，我们设备下有多条路径，第一条sdg 为group 1，第二个sdf为group 2，如果想切换主路径到sdd，那么需要执行：

RriyIrj.png!web

执行完毕后，可以执行quit 退出multipathd的交互，然后执行multipath -ll来确认主路径是否发生了变化。

有些时候，没有I/O流量的情况下，multipath -ll的输出并不会及时发生变化，不过没关系，引入一些IO，就可以看到active 路径发生了变化。

oot@host243:~# multipath -ll
2c2c5d201aba719ee dm-0 Bigtera ,VirtualStor_Scal
size=5.0T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 49:0:0:0 sdg 8:96 active ready running
|-+- policy='round-robin 0' prio=1 status=enabled
| `- 48:0:0:0 sdf 8:80 active ready running
|-+- policy='round-robin 0' prio=1 status=active
| `- 46:0:0:0 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 47:0:0:0 sde 8:64 active ready running

验证

对于分布式存储的iSCSI，每一条路径对应某一个存储Gateway，如何判断sdX和存储Gateway的映射关系呢？

root@host243:~# iscsiadm  -m session -P 3
iSCSI Transport Class version 2.0-870
version 2.0-873
Target: iqn.2020-01.lenovo:test
	Current Portal: 10.16.172.105:3260,1
	Persistent Portal: 10.16.172.105:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: <empty>
		Iface Netdev: <empty>
		SID: 16
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: <empty>
		password: ********
		username_in: <empty>
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 46	State: running
		scsi46 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdd		State: running
	Current Portal: 10.16.172.106:3260,1
	Persistent Portal: 10.16.172.106:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: <empty>
		Iface Netdev: <empty>
		SID: 17
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: <empty>
		password: ********
		username_in: <empty>
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 47	State: running
		scsi47 Channel 00 Id 0 Lun: 0
			Attached scsi disk sde		State: running
	Current Portal: 10.16.172.107:3260,1
	Persistent Portal: 10.16.172.107:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: <empty>
		Iface Netdev: <empty>
		SID: 18
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: <empty>
		password: ********
		username_in: <empty>
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 48	State: running
		scsi48 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdf		State: running
	Current Portal: 10.16.172.108:3260,1
	Persistent Portal: 10.16.172.108:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:f077d70d498
		Iface IPaddress: 10.16.172.243
		Iface HWaddress: <empty>
		Iface Netdev: <empty>
		SID: 19
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 30
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: <empty>
		password: ********
		username_in: <empty>
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 262144
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 49	State: running
		scsi49 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdg		State: running
root@host243:~#

通过上面的输出，可以看到 /dev/sdd对应的是10.16.172.105:3260 ，因此找到了对应关系。

修改完毕了主路径，如何从网路上判定已经发生了切换呢？

iftop -i bond0

通过iftop，查看网络流向，确定是否流向了正确的Gateway。

zmENNn6.png!web

前言

查看当前配置

切换主路径

验证

Recommend

Go学习笔记-面向对象struct之继承

告别2019：属于深度学习的十年，那些我们必须知道的经典

苹果供应商对2020年预期乐观：看好中国5G发展

林斌：小米自己的研发产线今日正式开工

我的2019年终总结（浴火重生）| 掘金年度征文

Flutter嵌套深？扩展函数了解一下

短视频2020：风继续吹，竞争还没有结束

制定2020年计划之前，先问问自己这10个问题

一位中年失业者的2019：这并不是最糟糕的生活，我不会认命

几百万的报销

About Joyk