原创文章，转载请注明出处
https://qiedd.com/
0. 前言
由于 VPS 的硬盘可能不太够，所以需要做一些文件系统上的压缩，因此在 ZFS 和 Btrfs 上取舍，最后选择了 ZFS
但是压缩方式上面让我犯了难，因此我们来测试一下一些比较优秀的压缩方式
https://github.com/openzfs/zfs
环境
腾讯云
系统： Arch Linux
CPU: Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz (2 Cores)
Memory: 4 GB
1. 一些命令
创建一个新的 mountpoint 来作为测试
# 创建mountpoint zfs create -o mountpoint=/test zroot/data/test # 查看压缩方式 zfs get compression # zroot/data/test compression zstd inherited from zroot # 查看压缩率 zroot/data/test compressratio 1.00x - # 安装iotop和fio pacman -Sy iotop fio # 写入测试 fio --name=seqwrite --rw=write --direct=1 --ioengine=psync --bs=128k --iodepth=32 --numjobs=10 --size=1G --group_reporting # 读取测试 fio --name=seqread --rw=read --direct=1 --ioengine=psync --bs=128k --iodepth=32 --numjobs=10 --size=1G --group_reporting # 查看进程io iotop -o # 生成一个随机的1G文件 dd if=/dev/urandom of=sample.txt bs=64M count=16
2. Lz4
写入
seqwrite: (groupid=0, jobs=10): err= 0: pid=3933293: Tue Jan 18 22:48:55 2022
write: IOPS=1117, BW=140MiB/s (146MB/s)(10.0GiB/73305msec); 0 zone resets
clat (usec): min=13, max=65950, avg=8856.39, stdev=1989.56
lat (usec): min=13, max=65951, avg=8857.77, stdev=1989.60
clat percentiles (usec):
| 1.00th=[ 19], 5.00th=[ 5604], 10.00th=[ 8291], 20.00th=[ 8979],
| 30.00th=[ 9110], 40.00th=[ 9110], 50.00th=[ 9241], 60.00th=[ 9241],
| 70.00th=[ 9241], 80.00th=[ 9372], 90.00th=[ 9634], 95.00th=[10028],
| 99.00th=[11469], 99.50th=[13304], 99.90th=[25035], 99.95th=[29754],
| 99.99th=[39584]
bw ( KiB/s): min=129174, max=827864, per=100.00%, avg=144240.45, stdev=6509.22, samples=1445
iops : min= 1006, max= 6464, avg=1126.41, stdev=50.83, samples=1445
lat (usec) : 20=1.12%, 50=0.78%, 100=0.02%, 250=0.09%, 500=0.21%
lat (usec) : 750=0.23%, 1000=0.05%
lat (msec) : 2=0.33%, 4=1.07%, 10=90.76%, 20=5.18%, 50=0.17%
lat (msec) : 100=0.01%
cpu : usr=0.07%, sys=0.61%, ctx=82572, majf=8, minf=138
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,81920,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=140MiB/s (146MB/s), 140MiB/s-140MiB/s (146MB/s-146MB/s), io=10.0GiB (10.7GB), run=73305-73305msec读取
seqread: (groupid=0, jobs=10): err= 0: pid=3941480: Tue Jan 18 22:54:13 2022
read: IOPS=1222, BW=153MiB/s (160MB/s)(10.0GiB/67020msec)
clat (usec): min=15, max=147529, avg=6833.55, stdev=18975.55
lat (usec): min=15, max=147529, avg=6833.68, stdev=18975.54
clat percentiles (usec):
| 1.00th=[ 35], 5.00th=[ 46], 10.00th=[ 56], 20.00th=[ 63],
| 30.00th=[ 64], 40.00th=[ 66], 50.00th=[ 71], 60.00th=[ 77],
| 70.00th=[ 83], 80.00th=[ 103], 90.00th=[ 39584], 95.00th=[ 62129],
| 99.00th=[ 74974], 99.50th=[ 78119], 99.90th=[ 84411], 99.95th=[ 88605],
| 99.99th=[119014]
bw ( MiB/s): min= 1863, max= 2270, per=100.00%, avg=1982.82, stdev= 9.51, samples=1111
iops : min=14905, max=18163, avg=15861.64, stdev=76.03, samples=1111
lat (usec) : 20=0.30%, 50=7.63%, 100=71.39%, 250=5.27%, 500=0.18%
lat (usec) : 750=0.10%, 1000=0.12%
lat (msec) : 2=0.37%, 4=0.49%, 10=2.79%, 20=0.40%, 50=2.29%
lat (msec) : 100=8.65%, 250=0.02%
cpu : usr=0.03%, sys=1.16%, ctx=13494, majf=0, minf=456
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=81920,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=153MiB/s (160MB/s), 153MiB/s-153MiB/s (160MB/s-160MB/s), io=10.0GiB (10.7GB), run=67020-67020msec3. Gzip
写入
seqwrite: (groupid=0, jobs=10): err= 0: pid=3943106: Tue Jan 18 23:02:32 2022
write: IOPS=189, BW=23.7MiB/s (24.8MB/s)(10.0GiB/432137msec); 0 zone resets
clat (usec): min=12, max=98520, avg=52338.47, stdev=11922.03
lat (usec): min=13, max=98522, avg=52340.13, stdev=11922.07
clat percentiles (usec):
| 1.00th=[ 23], 5.00th=[26346], 10.00th=[44827], 20.00th=[52167],
| 30.00th=[53216], 40.00th=[53740], 50.00th=[54264], 60.00th=[55313],
| 70.00th=[57410], 80.00th=[58459], 90.00th=[60031], 95.00th=[62129],
| 99.00th=[65799], 99.50th=[67634], 99.90th=[72877], 99.95th=[74974],
| 99.99th=[86508]
bw ( KiB/s): min=17912, max=729856, per=100.00%, avg=24460.45, stdev=2562.57, samples=8565
iops : min= 138, max= 5702, avg=190.90, stdev=20.02, samples=8565
lat (usec) : 20=0.69%, 50=1.27%, 100=0.02%, 250=0.02%, 500=0.10%
lat (usec) : 750=0.18%, 1000=0.06%
lat (msec) : 2=0.23%, 4=0.34%, 10=0.43%, 20=0.62%, 50=9.44%
lat (msec) : 100=86.58%
cpu : usr=0.02%, sys=0.13%, ctx=83526, majf=0, minf=126
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,81920,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=23.7MiB/s (24.8MB/s), 23.7MiB/s-23.7MiB/s (24.8MB/s-24.8MB/s), io=10.0GiB (10.7GB), run=432137-432137msec读取
seqread: (groupid=0, jobs=10): err= 0: pid=3953413: Tue Jan 18 23:14:01 2022
read: IOPS=1140, BW=143MiB/s (149MB/s)(10.0GiB/71840msec)
clat (usec): min=28, max=139673, avg=8034.70, stdev=19823.39
lat (usec): min=28, max=139673, avg=8034.87, stdev=19823.38
clat percentiles (usec):
| 1.00th=[ 40], 5.00th=[ 53], 10.00th=[ 62], 20.00th=[ 63],
| 30.00th=[ 66], 40.00th=[ 74], 50.00th=[ 78], 60.00th=[ 82],
| 70.00th=[ 91], 80.00th=[ 137], 90.00th=[ 46924], 95.00th=[ 62129],
| 99.00th=[ 76022], 99.50th=[ 79168], 99.90th=[ 86508], 99.95th=[ 88605],
| 99.99th=[107480]
bw ( KiB/s): min=30095, max=1108520, per=100.00%, avg=158744.43, stdev=18988.74, samples=1310
iops : min= 234, max= 8657, avg=1239.78, stdev=148.32, samples=1310
lat (usec) : 50=4.30%, 100=70.70%, 250=5.73%, 500=0.36%, 750=0.20%
lat (usec) : 1000=0.21%
lat (msec) : 2=0.52%, 4=0.62%, 10=2.78%, 20=0.88%, 50=4.53%
lat (msec) : 100=9.16%, 250=0.02%
cpu : usr=0.04%, sys=1.11%, ctx=17249, majf=0, minf=439
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=81920,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=143MiB/s (149MB/s), 143MiB/s-143MiB/s (149MB/s-149MB/s), io=10.0GiB (10.7GB), run=71840-71840msec4. Zstd
写入
seqwrite: (groupid=0, jobs=10): err= 0: pid=3955112: Tue Jan 18 23:17:19 2022
write: IOPS=556, BW=69.6MiB/s (73.0MB/s)(10.0GiB/147117msec); 0 zone resets
clat (usec): min=13, max=54822, avg=17787.35, stdev=3391.08
lat (usec): min=13, max=54824, avg=17789.17, stdev=3391.04
clat percentiles (usec):
| 1.00th=[ 21], 5.00th=[15401], 10.00th=[16712], 20.00th=[17695],
| 30.00th=[17957], 40.00th=[18220], 50.00th=[18220], 60.00th=[18482],
| 70.00th=[18744], 80.00th=[19006], 90.00th=[19792], 95.00th=[20317],
| 99.00th=[21890], 99.50th=[22414], 99.90th=[24249], 99.95th=[25560],
| 99.99th=[30016]
bw ( KiB/s): min=56576, max=726212, per=100.00%, avg=71873.44, stdev=4721.09, samples=2906
iops : min= 442, max= 5672, avg=561.34, stdev=36.88, samples=2906
lat (usec) : 20=0.98%, 50=0.93%, 100=0.01%, 250=0.09%, 500=0.15%
lat (usec) : 750=0.15%, 1000=0.06%
lat (msec) : 2=0.15%, 4=0.37%, 10=0.96%, 20=88.67%, 50=7.47%
lat (msec) : 100=0.01%
cpu : usr=0.06%, sys=0.41%, ctx=84403, majf=0, minf=119
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,81920,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
WRITE: bw=69.6MiB/s (73.0MB/s), 69.6MiB/s-69.6MiB/s (73.0MB/s-73.0MB/s), io=10.0GiB (10.7GB), run=147117-147117msec读取
seqread: (groupid=0, jobs=10): err= 0: pid=3964126: Tue Jan 18 23:21:49 2022
read: IOPS=1164, BW=146MiB/s (153MB/s)(10.0GiB/70371msec)
clat (usec): min=27, max=151967, avg=7762.81, stdev=20941.61
lat (usec): min=27, max=151967, avg=7762.94, stdev=20941.61
clat percentiles (usec):
| 1.00th=[ 38], 5.00th=[ 48], 10.00th=[ 61], 20.00th=[ 63],
| 30.00th=[ 64], 40.00th=[ 65], 50.00th=[ 71], 60.00th=[ 77],
| 70.00th=[ 84], 80.00th=[ 104], 90.00th=[ 50070], 95.00th=[ 67634],
| 99.00th=[ 80217], 99.50th=[ 83362], 99.90th=[ 89654], 99.95th=[ 94897],
| 99.99th=[124257]
bw ( KiB/s): min=20996, max=1533480, per=100.00%, avg=169112.61, stdev=31295.51, samples=1263
iops : min= 164, max=11976, avg=1320.63, stdev=244.47, samples=1263
lat (usec) : 50=5.66%, 100=73.24%, 250=5.37%, 500=0.12%, 750=0.08%
lat (usec) : 1000=0.07%
lat (msec) : 2=0.25%, 4=0.39%, 10=2.83%, 20=0.36%, 50=1.57%
lat (msec) : 100=10.00%, 250=0.04%
cpu : usr=0.04%, sys=1.04%, ctx=13941, majf=0, minf=460
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=81920,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: bw=146MiB/s (153MB/s), 146MiB/s-146MiB/s (153MB/s-153MB/s), io=10.0GiB (10.7GB), run=70371-70371msec4. 压缩率
测试文件：PostgreSQL 1G 数据文件
# lz4 zroot/data/test compressratio 2.54x - # gzip zroot/data/test compressratio 3.33x - # zstd zroot/data/test compressratio 3.51x -
5. 结论
如果对性能有要求，建议在 lz4 和 zstd 中做取舍
zstd 在高压缩率的情况下，写入性能也比 gzip 高，确实不错👍
0 条评论