<服务>Rsync

时间:Nov. 27, 2016 分类:

目录:

Rsync

rsync是一款开源的,快速的,多功能的,本地或者远程数据同步,同理的还有drbd,用于大数据量,通过底层文件系统进行同步。

Rsync官网

优势

可以实现类似scp,cp和rm的功能,另外因为其独特的quick check算法,可以同步大小或者最后修改时间放生变化的文件或者目录,也可以根据权限,所属组等属性的变化进行同步,不同于scp和cp等功能,可以实现增量拷贝的功能。

rsync的三种工作方式

  • 单主机本地传输数据
  • 借助rcp,ssh等通信渠道来传输数据
  • 以守护进程的方式传输数据

    安装方式

[root@sersync ~]# yum install -y rsync

使用方式

SYNOPSIS
       Local:  rsync [OPTION...] SRC... [DEST]

       Access via remote shell:
         Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
         Push: rsync [OPTION...] SRC... [USER@]HOST:DEST

       Access via rsync daemon:
         Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
               rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
         Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
               rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST

Local模式

类似cp

[root@sersync ~]# man rsync
[root@sersync ~]# rsync /etc/hosts /tmp
[root@sersync ~]# ll /tmp/
总用量 52
-rw-r--r--  1 root root  198 11月 26 13:41 hosts

类似cp -a(dpr) 复制所有文件和对应权限

[root@sersync tmp]# rsync -avz /etc/hosts /tmp
sending incremental file list
hosts

sent 149 bytes  received 31 bytes  360.00 bytes/sec
total size is 198  speedup is 1.10

类似rm

[root@sersync tmp]# mkdir data
[root@sersync tmp]# touch data/a
[root@sersync tmp]# mkdir /null
[root@sersync tmp]# rsync -r --delete /null/ /data/
[root@sersync tmp]# ll data
总用量 0

原理就是/null/目录为空,同步到data/也为空

Access via remote shell模式

借助ssh通道

借助ssh通道远程推送

[root@sersync tmp]# rsync  -e 'ssh -p22 ' /etc/hosts root@192.168.0.183:~
root@192.168.0.183's password: 

[root@nfs ~]# ll
总用量 128
-rw-r--r--  1 root root   198 11月 26 15:03 hosts

借助ssh通道远程拉取

[root@sersync ~]# rsync -e 'ssh -p22' root@192.168.0.183:/root/hosts .
root@192.168.0.183's password: 
[root@sersync ~]# ll
总用量 128
-rw-r--r--. 1 root root     3 10月 12 23:54 a
-rw-------. 1 root root  1757 10月 12 20:22 anaconda-ks.cfg
-rw-r--r--  1 root root   198 11月 26 15:20 hosts

常用参数

  • -v 输出传输的详细信息
  • -z 传输的时候进行压缩,提高传输速率
  • -a 递归方式传输,并保持文件属性,等于-rtopgDl
  • -r 递归方式
  • -t 保持文件时间信息
  • -o 保持文件所属主信息
  • -p 保持文件权限
  • -g 保持文件所属组信息
  • -D 保持设备文件信息
  • -l 保留软链
  • -e 制定通道
  • -bwlimit 限速

Access via rsync daemon模式

server端配置文件

配置文件为/etc/rsyncd.conf,默认情况下不存在。

[root@sersync ~]# vi /etc/rsyncd.conf
uid = rsync                             
gid = rsync
use chroot = no                         # 安全性
max connections = 200                   # 同时连接客户端数量
timeout = 300                           # 超时时间
pid file = /var/run/rsyncd.pid          
lock file = /var/run/rsync.lock         # 锁
log file = /var/log/rsyncd.log
read only = false                       # 只读关闭  
[why]                                   # 模块
path = /data/ 
ignore errors                           # 自动忽略错误
read only = false                       # 可写
list = false                            # 不允许列表
hosts allow = 10.0.0.0/24               # 允许主机网段
hosts deny = 0.0.0.0/32                 # 拒绝主机网段
auth users = rsync_backup               # 虚拟用户
secrets file = /etc/rsync.password      # 密码文件

尽量配置文件中不要写中文

cat /etc/rsyncd.conf
uid = rsync             
gid = rsync
use chroot = no             
max connections = 200       
timeout = 300       
pid file = /var/run/rsyncd.pid      
lock file = /var/run/rsync.lock 
log file = /var/log/rsyncd.log      
[why]           
path = /why/                
ignore errors   
read only = false
list = false        
hosts allow = 192.168.0.0
hosts deny = 0.0.0.0/32 
auth users = rsync_backup
secrets file = /etc/rsync.password

启动rsync daemon

写入虚拟用户密码
[root@sersync ~]# echo 'rsync_backup:123456' > /etc/rsync.password
[root@sersync ~]# chmod 600 /etc/rsync.password
[root@sersync ~]# useradd rsync -s /sbin/nologin -M
[root@sersync ~]# mkdir /why
[root@sersync ~]# chown -R rsync.rsync /why

[root@sersync ~]# rsync --daemon
[root@sersync ~]# netstat -nlput | grep 873
tcp        0      0 0.0.0.0:873                 0.0.0.0:*                   LISTEN      3384/rsync          
tcp        0      0 :::873                      :::*                        LISTEN      3384/rsync          
[root@sersync ~]# ps -ef | grep rsync
root      3384     1  0 17:32 ?        00:00:00 rsync --daemon
root      3392  2653  0 17:35 pts/0    00:00:00 grep rsync

相关参数和命令

--port=PORT         指定端口,不使用默认的873
--config=FLIE       指定配置文件路径
--address           绑定指定IP地址提供服务
pkill rsync         关闭rsync

client端

[root@nfs ~]# echo '123456' > /etc/rsync.password
[root@nfs ~]# chmod 600 /etc/rsync.password 

client端只配置密码

传输数据

拉取数据

[root@nfs ~]# rsync -avz rsync_backup@192.168.0.184::why /data1 
Password: 
receiving incremental file list
created directory /data1
./
w/
w/h/
w/h/y/

sent 76 bytes  received 161 bytes  67.71 bytes/sec
total size is 0  speedup is 0.00

这个why是模块名称,需要输入的密码为rsync_backup的密码,免密码需要制定密码文件

[root@nfs ~]# rsync -avz rsync_backup@192.168.0.184::why /data1 --password-file=/etc/rsync.password 
receiving incremental file list

sent 64 bytes  received 149 bytes  142.00 bytes/sec
total size is 0  speedup is 0.00

推送数据

[root@nfs data1]# touch {1..5}
[root@nfs data1]# ll
总用量 4
-rw-r--r-- 1 root root    0 11月 26 18:35 1
-rw-r--r-- 1 root root    0 11月 26 18:35 2
-rw-r--r-- 1 root root    0 11月 26 18:35 3
-rw-r--r-- 1 root root    0 11月 26 18:35 4
-rw-r--r-- 1 root root    0 11月 26 18:35 5
drwxr-xr-x 3 root root 4096 11月 26 18:30 w
[root@nfs data1]# rsync -avz /data1/ rsync_backup@192.168.0.184::why --password-file=/etc/rsync.password 
sending incremental file list
./
1
2
3
4
5

sent 297 bytes  received 109 bytes  270.67 bytes/sec
total size is 0  speedup is 0.00

其他格式

[root@nfs data1]# rsync -avz /data1/ rsync://rsync_backup@192.168.0.184/why --password-file=/etc/rsync.password 
sending incremental file list

sent 114 bytes  received 11 bytes  250.00 bytes/sec
total size is 0  speedup is 0.00

文件排除

client端排除

exclude方式

删除服务端why模块目录下文件 [root@sersync why]# rm -fr * 排除a文件推送

[root@nfs data1]# rsync -avz /data1/ --exclude=1 rsync://rsync_backup@192.168.0.184/why --password-file=/etc/rsync.password 
sending incremental file list
./
2
3
4
5
w/
w/h/
w/h/y/

sent 264 bytes  received 99 bytes  55.85 bytes/sec
total size is 0  speedup is 0.00

可以看到推送过程中没有a,可以在server端查看

[root@sersync why]# ls a
ls: 无法访问a: 没有那个文件或目录

排除多个文件

[root@nfs data1]# rsync -avz /data1/ --exclude={1,2} rsync://rsync_backup@192.168.0.184/why --password-file=/etc/rsync.password 
sending incremental file list
./
3
4
5
w/
w/h/
w/h/y/

sent 222 bytes  received 80 bytes  54.91 bytes/sec
total size is 0  speedup is 0.00
[root@nfs data1]# rsync -avz /data1/ --exclude={1..3} rsync://rsync_backup@192.168.0.184/why --password-file=/etc/rsync.password 
sending incremental file list
./
4
5
w/
w/h/
w/h/y/

sent 180 bytes  received 61 bytes  482.00 bytes/sec
total size is 0  speedup is 0.00

exclude-from方式

[root@nfs data1]# rsync -avz /data1/ --exclude-from=/a rsync://rsync_backup@192.168.0.184/why --password-file=/etc/rsync.password 
sending incremental file list
./
w/
w/h/
w/h/y/

sent 80 bytes  received 23 bytes  206.00 bytes/sec
total size is 0  speedup is 0.00

server端排除

exclude方式

修改配置文件rsyncd.conf, 添加exclude = 1 2 w/h

[root@sersync why]# pkill `cat /var/run/rsyncd.pid`
[root@sersync why]# rsync --daemon
[root@sersync why]# failed to create pid file /var/run/rsyncd.pid: File exists
^C
[root@sersync why]# rm -f /var/run/rsyncd.pid
[root@sersync why]# rsync --daemon

client端推送数据

[root@nfs data1]# rsync -avz /data1/ rsync://rsync_backup@192.168.0.184/why --password-file=/etc/rsync.password 
sending incremental file list
./
skipping daemon-excluded file "1"
skipping daemon-excluded file "2"
3
4
5
w/
skipping daemon-excluded directory "w/h"
*** Skipping any contents from this failed directory ***

sent 228 bytes  received 74 bytes  604.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]

一般不会使用这种方式。

无差异同步

客户端和服务端的目录文件

[root@nfs data1]# ll
总用量 4
-rw-r--r-- 1 root root    0 11月 26 18:35 1
-rw-r--r-- 1 root root    0 11月 26 18:35 2
-rw-r--r-- 1 root root    0 11月 26 18:35 3
-rw-r--r-- 1 root root    0 11月 26 18:35 4
-rw-r--r-- 1 root root    0 11月 26 18:35 5
drwxr-xr-x 3 root root 4096 11月 26 18:30 w

[root@sersync why]# ll
总用量 0

客户端同步服务端

[root@nfs data1]# rsync -avz --delete rsync_backup@192.168.0.184::why /data1 --password-file=/etc/rsync.password 
receiving incremental file list
deleting w/h/y/
deleting w/h/
deleting w/
deleting 5
deleting 4
deleting 3
deleting 2
deleting 1
./

sent 64 bytes  received 112 bytes  352.00 bytes/sec
total size is 0  speedup is 0.00

我们可以看到因为服务端的模块目录下没有文件,导致了同步到客户端时删除了客户端的文件,推送也是同理。

应用场景

不需要实时同步的同步需求。

多模块配置

多模块配置文件

uid = rsync
gid = rsync
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 192.168.0.0
hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password
exclude-from = /a
[why]
path = /why/
[mabiao]                
path = /mabiao/ 
[root@sersync why]# mkdir /mabiao
[root@sersync why]# chown rsync.rsync /mabiao

重启rsync服务

[root@nfs data1]# rsync -avz  /data1/ rsync_backup@192.168.0.184::why  --password-file=/etc/rsync.password 
sending incremental file list
./
a

sent 75 bytes  received 30 bytes  70.00 bytes/sec
total size is 0  speedup is 0.00

[root@nfs data1]# rsync -avz  /data1/ rsync_backup@192.168.0.184::mabiao  --password-file=/etc/rsync.password 
sending incremental file list
./
a

sent 75 bytes  received 30 bytes  70.00 bytes/sec
total size is 0  speedup is 0.00

详解

配置文件中写到模块上边的是模块公有配置内容,模板内的为模块私有配置文件。

可能会遇到的错误

服务端模块目录没有权限

[root@nfs data1]# rsync -avz  /data1/ rsync_backup@192.168.0.184::mabiao  --password-file=/etc/rsync.password 
sending incremental file list
./
rsync: failed to set times on "." (in mabiao): Operation not permitted (1)
a
rsync: mkstemp ".a.DCm8Tl" (in mabiao) failed: Permission denied (13)

sent 75 bytes  received 30 bytes  19.09 bytes/sec
total size is 0  speedup is 0.00

服务端服务没有启动

[root@nfs data1]# rsync -avz --delete rsync_backup@192.168.0.184::why /data1 --password-file=/etc/rsync.password 
rsync: failed to connect to 192.168.0.184: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(124) [receiver=3.0.6]