Redis-集群伸缩
标签:Redis

集群伸缩

1. 伸缩原理

伸就是加入新的节点,缩就是将节点移除

集群伸缩就相当于槽和数据在节点之前的移动

2. 扩容集群

Step 1: 准备新节点

redis-server conf/redis-6385.conf

Step 2: 加入集群

127.0.0.1:6379> cluster meet 127.0.0.1 6385

加入集群的作用:

或者利用 redis-trib.rb 加入集群

redis-trib.rb add-node new_host:new_port existing_host:existing_port --slave --master-id <arg>

redis-trib.rb add-node 127.0.0.1:6385 127.0.0.1:6379

建议使用redis-trib.rb能够避免新节点在这之前已经加入过其他集群,而造成故障。

Step 3:迁移槽和数据

槽迁移计划:

原来是5641/5640个槽,新的节点加入之后,每个槽拿出5641-4096个新的节点

迁移数据:

  1. 对目标节点发送: cluster setslot {slot} importing {sourceNodeId} 命令,让目标节点准备导入槽的数据。
  2. 对源节点发送:clsuter setslot {slot} migrating {targetNodeId}命令,让源节点准备迁出槽的数据。
  3. 源节点循环执行:cluster getkeysinslot {slot} {count}命令,每次获取count个属于槽的键。
  4. 在源节点上执行:migrate {targetIp} {targetPort} key 0 {timeout} 命令,把指定key迁移
  5. 重复执行步骤3~4直到槽下所有的键数据迁移到目标节点。
  6. 向集群内所有主节点发送 cluster setslot {slot} node {targetNodeId}命令,通知槽分配给目标节点。

上面是数据迁移的流程图

2.1 测试

Step 1:先启动7000到7005的节点,并分配相应的主从关系

Step 2:复制conf文件

sed 's/7000/7006/g' redis-7000.conf > redis-7006.conf
sed 's/7000/7007/g' redis-7000.conf > redis-7007.conf

Step 3: 启动7006、7007

[root@localhost redis]# ./bin/redis-server ./cluster/redis-7006.conf 
[root@localhost redis]# ./bin/redis-server ./cluster/redis-7007.conf 

Step 4:加入到集群中

[root@localhost redis]# ./bin/redis-cli -p 7000 cluster meet 127.0.0.1 7006
OK
[root@localhost redis]# ./bin/redis-cli -p 7000 cluster meet 127.0.0.1 7007
OK

Step 5:7000 slave of 7006

[root@localhost redis]# ./bin/redis-cli -p 7000 cluster nodes
ba2c1179961f12edc4ea22a46674bb0e4fda1abb 127.0.0.1:7003@17003 slave dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0 0 1529758996774 4 connected
b9d7746098147ceb8614020b51edb5f768b32c06 127.0.0.1:7006@17006 master - 0 1529758995000 0 connected
b45227eb8412073c644c724e6b9fedf207e4aefa 127.0.0.1:7005@17005 slave 98cdd9928767f879cdc39453c54f5249beecc35e 0 1529758997780 6 connected
45c2e0c0505e1347f9d4c4dca64a361823edf901 127.0.0.1:7004@17004 slave 4472f9374bb20b40f5b65d781bf33410bc7d5386 0 1529758995769 5 connected
4472f9374bb20b40f5b65d781bf33410bc7d5386 127.0.0.1:7002@17002 master - 0 1529758995000 3 connected 10923-16383
898cc409305502ad7d37f5c7cb4676ed123521bd 127.0.0.1:7007@17007 master - 0 1529758996000 7 connected
98cdd9928767f879cdc39453c54f5249beecc35e 127.0.0.1:7000@17000 myself,master - 0 1529758995000 1 connected 0-5460
dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0 127.0.0.1:7001@17001 master - 0 1529758998784 2 connected 5461-10922
[root@localhost redis]# ./bin/redis-cli -p 7007 cluster replicate b9d7746098147ceb8614020b51edb5f768b32c06

Step 6:使用redis-trib.rb迁移数据

[root@localhost redis]# ./src/redis-trib.rb reshard 127.0.0.1:7000

Step 7:查看结果

[root@localhost redis]# ./bin/redis-cli -p 7000 cluster slots
1) 1) (integer) 0
   2) (integer) 1364
   3) 1) "127.0.0.1"
      2) (integer) 7006
      3) "b9d7746098147ceb8614020b51edb5f768b32c06"
   4) 1) "127.0.0.1"
      2) (integer) 7007
      3) "898cc409305502ad7d37f5c7cb4676ed123521bd"
2) 1) (integer) 5461
   2) (integer) 6826
   3) 1) "127.0.0.1"
      2) (integer) 7006
      3) "b9d7746098147ceb8614020b51edb5f768b32c06"
   4) 1) "127.0.0.1"
      2) (integer) 7007
      3) "898cc409305502ad7d37f5c7cb4676ed123521bd"
3) 1) (integer) 10923
   2) (integer) 12287
   3) 1) "127.0.0.1"
      2) (integer) 7006
      3) "b9d7746098147ceb8614020b51edb5f768b32c06"
   4) 1) "127.0.0.1"
      2) (integer) 7007
      3) "898cc409305502ad7d37f5c7cb4676ed123521bd"
4) 1) (integer) 12288
   2) (integer) 16383
   3) 1) "127.0.0.1"
      2) (integer) 7002
      3) "4472f9374bb20b40f5b65d781bf33410bc7d5386"
   4) 1) "127.0.0.1"
      2) (integer) 7004
      3) "45c2e0c0505e1347f9d4c4dca64a361823edf901"
5) 1) (integer) 1365
   2) (integer) 5460
   3) 1) "127.0.0.1"
      2) (integer) 7000
      3) "98cdd9928767f879cdc39453c54f5249beecc35e"
   4) 1) "127.0.0.1"
      2) (integer) 7005
      3) "b45227eb8412073c644c724e6b9fedf207e4aefa"
6) 1) (integer) 6827
   2) (integer) 10922
   3) 1) "127.0.0.1"
      2) (integer) 7001
      3) "dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0"
   4) 1) "127.0.0.1"
      2) (integer) 7003
      3) "ba2c1179961f12edc4ea22a46674bb0e4fda1abb"

3. 收缩集群

下线槽:

忘记节点:

redis-cli>cluster forget {downNodeId}

该命令是60s有效的,如果60s内,仍有节点没有忘记它,则会继续扩散这个消息。

3.1 测试

将上面加入的7006和7007下线

Step 1:迁移槽

[root@localhost redis]# ./src/redis-trib.rb reshard --from b9d7746098147ceb8614020b51edb5f768b32c06 --to 98cdd9928767f879cdc39453c54f5249beecc35e --slots 1365 127.0.0.1:7006
[root@localhost redis]# ./src/redis-trib.rb reshard --from b9d7746098147ceb8614020b51edb5f768b32c06 --to dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0 --slots 1366 127.0.0.1:7006
[root@localhost redis]# ./src/redis-trib.rb reshard --from b9d7746098147ceb8614020b51edb5f768b32c06 --to 4472f9374bb20b40f5b65d781bf33410bc7d5386 --slots 1365 127.0.0.1:7006

最后查看:

[root@localhost redis]# ./bin/redis-cli -p 7000 cluster nodes45c2e0c0505e1347f9d4c4dca64a361823edf901 127.0.0.1:7004@17004 slave 4472f9374bb20b40f5b65d781bf33410bc7d5386 0 1529808221682 11 connected
ba2c1179961f12edc4ea22a46674bb0e4fda1abb 127.0.0.1:7003@17003 slave dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0 0 1529808225705 10 connected
b45227eb8412073c644c724e6b9fedf207e4aefa 127.0.0.1:7005@17005 slave 98cdd9928767f879cdc39453c54f5249beecc35e 0 1529808221000 9 connected
98cdd9928767f879cdc39453c54f5249beecc35e 127.0.0.1:7000@17000 myself,master - 0 1529808224000 9 connected 0-5460
898cc409305502ad7d37f5c7cb4676ed123521bd 127.0.0.1:7007@17007 slave 4472f9374bb20b40f5b65d781bf33410bc7d5386 0 1529808223695 11 connected
4472f9374bb20b40f5b65d781bf33410bc7d5386 127.0.0.1:7002@17002 master - 0 1529808223000 11 connected 10923-16383
dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0 127.0.0.1:7001@17001 master - 0 1529808224699 10 connected 5461-10922
b9d7746098147ceb8614020b51edb5f768b32c06 127.0.0.1:7006@17006 master - 0 1529808225000 8 connected

可见7006上已经没有了槽了。

Step 2:删除(下线)节点

[root@localhost redis]# ./src/redis-trib.rb del-node 127.0.0.1:7000 898cc409305502ad7d37f5c7cb4676ed123521bd
>>> Removing node 898cc409305502ad7d37f5c7cb4676ed123521bd from cluster 127.0.0.1:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
[root@localhost redis]# ./src/redis-trib.rb del-node 127.0.0.1:7000 b9d7746098147ceb8614020b51edb5f768b32c06
>>> Removing node b9d7746098147ceb8614020b51edb5f768b32c06 from cluster 127.0.0.1:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

注意要下线从节点,再下线主节点,避免触发主节点的故障转移。

Step 3:查看

[root@localhost redis]# ./bin/redis-cli -p 7000 cluster nodes45c2e0c0505e1347f9d4c4dca64a361823edf901 127.0.0.1:7004@17004 slave 4472f9374bb20b40f5b65d781bf33410bc7d5386 0 1529808538000 11 connected
ba2c1179961f12edc4ea22a46674bb0e4fda1abb 127.0.0.1:7003@17003 slave dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0 0 1529808538076 10 connected
b45227eb8412073c644c724e6b9fedf207e4aefa 127.0.0.1:7005@17005 slave 98cdd9928767f879cdc39453c54f5249beecc35e 0 1529808538000 9 connected
98cdd9928767f879cdc39453c54f5249beecc35e 127.0.0.1:7000@17000 myself,master - 0 1529808537000 9 connected 0-5460
4472f9374bb20b40f5b65d781bf33410bc7d5386 127.0.0.1:7002@17002 master - 0 1529808538000 11 connected 10923-16383
dfbb938a4e1a3b031d6a5fdf4955ec9ae23c9ff0 127.0.0.1:7001@17001 master - 0 1529808539083 10 connected 5461-10922

可见上面的7006和7007已经下线了。

  • 6 min read

CONTRIBUTORS


  • 6 min read