0

定位磁盘IO问题

 9 months ago
source link: https://zzyongx.github.io/blogs/diskio-troubleshooting.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

1 磁盘write吞吐饱和的表现

磁盘write会先写page cache，page cache标记为Dirty（不可直接回收），然后由kswapd进程把page cache写入磁盘，page cache标记为Clean（可回收）。

如果write吞吐饱和，Dirty page的数量（大小）会较大，或持续上升。 

# 物理机查看
for ((i=0; i<100; i++)); do echo $(date +%T) $(cat /proc/meminfo | grep Dirty); done

# 容器查看
for ((i=0; i<100; i++)); do echo $(date +%T) $(cat /sys/fs/cgroup/memory/memory.stat | grep "^dirty"); done

磁盘通常是读优先，可以通过wbt配置，调整写的优先级。 

# sdb 是相应的磁盘，可用通过df查看
cat /sys/block/sdb/queue/wbt_lat_usec
75000

echo 0 | sudo tee /sys/block/sdb/queue/wbt_lat_usec

下面两个参数，控制dirty page的写入 

# dirty page 存在超过30S，会写入磁盘
vm.dirty_expire_centisecs = 3000

# dirty page的大小超过available内存的20%，开始写入磁盘
vm.dirty_ratio = 20

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK