在Linux系统中将SSD当块设备缓存的方法

2015-07-02 15:58:17 脚本之家

原理

写操作先缓存到ssd硬盘上，然后通过一定策略写到普通硬盘上；

读操作热点数据可以缓存到ssd硬盘上，提高读取数据的速度。

软件 ssd 缓存开源软件主要有bcache ，flashcache

bcache 需要编译最新的内核，要求比较高，配置比较复杂，目前主要用于测试环境；

flashcache 是facebook内部使用的ssd cache软件，开源出来供大家使用，目前有人编译出rhel6的rpm包，在centos6上可以直接安装使用。

本次初步测试了flashcache

测试环境

sdc为 ssd 硬盘 160G*4 raid 10 用fio测试 4k 写iops为 5433

sdb为普通sas硬盘 146G*2 raid 1 用fio测试 4k 写iops为 447

初步简单测试结果：

4k 随机写可以达到 5014

4k随机读可以达到45874

flashcache 安装配置方法

安装

rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm yum install flashcache-utils kmod-flashcache

配置:

用sdc 做sdb的缓存

flashcache_create -p back cachedev /dev/sdc /dev/sdb cachedev cachedev, ssd_devname /dev/sdc, disk_devname /dev/sdb cache mode WRITE_BACK block_size 8, md_block_size 8, cache_size 0 Flashcache metadata will use 1192MB of your 24016MB main memory

可以用3种方式：

Writethrough - safest, all writes are cached to ssd but also written to disk immediately. If your ssd has slower write performance than your disk (likely for early generation SSDs purchased in 2008-2010), this may limit your system write performance. All disk reads are cached (tunable).

数据同时写到ssd和普通硬盘

Writearound - again, very safe, writes are not written to ssd but directly to disk. Disk blocks will only be cached after they are read. All disk reads are cached (tunable).

数据同时绕过ssd，直接写到普通硬盘

Writeback - fastest but less safe. Writes only go to the ssd initially, and based on various policies are written to disk later. All disk reads are cached (tunable).

数据先写到ssd，随后写到普通硬盘

查看

flashstat

======================================================================================================

Flashstat: a tool for flashcache status per second

Author : NinGoo(seaman.ning@gmail.com)

Version : 0.3

======================================================================================================

SSD Device: /dev/sdc Disk Device: /dev/sdb Cache Mode: WRITE_BACK

Capacity: 303998M Block Size: 4K Meta Block Size: 4096b

Total Blocks: 77823488 Cached Blocks: 9 Cached Percent: 0

Set Numbers: 512 Dirty Blocks: 0 Dirty Percent: 0

cache_all: 1 reclaim_policy: FIFO dirty_thresh_pct: 20

max_clean_ios_set: 2 max_clean_ios_total: 4 skip_seq_thresh: 0K

======================================================================================================

time read/s write/s diskr/s diskw/s ssdr/s ssdw/s uread/s uwrit/s metaw/s clean/s repl/s wrepl/s hit% whit% dwhit%

03-13 14:51:20 0 0 0 0 0 0 0 0 0 0 0 0 0|10 0|0 0|0

fdisk -l /dev/mapper/cachedev

可以看到一个设备

Disk /dev/mapper/cachedev: 146.8 GB, 146778685440 bytes

255 heads, 63 sectors/track, 17844 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x0002da0a

Device Boot Start End Blocks Id System

使用：

可以当一个块设备来使用，划分vg lv

pvcreate /dev/mapper/cachedev Physical volume "/dev/mapper/cachedev" successfully created vgcreate ssdcachetest /dev/mapper/cachedev Volume group "ssdcachetest" successfully created lvcreate -L 100G -n ssdcache_test_lv1 ssdcachetest Logical volume "ssdcache_test_lv1" created

删除

dmsetup remove cachedev flashcache_destroy /dev/sdc