<博客建立历程>阿里云建站19——网站迁移

时间:Aug. 14, 2017 分类:

目录:

whysdomain网站迁移

网站迁移的主要原因是当时购买阿里云主机的时候,挂载的磁盘无法卸载,导致每月维护成本较高,所以重新开一台阿里云主机进行配置,以前在上边做的一些操作对主机环境影响也较大,此次对主机进行规划。

计划为

  1. 初步配置主机
  2. 使用docker来维护站点所有信息和进行一些其他操作,进而隔离所有的环境,下次如果迁移,导出docker容器。
  3. 图片提供统一通过七牛云来进行图片存储和展示
  4. (日后)重构blog代码,升级Django版本

基础配置

关闭防火墙和selinux

/etc/init.d/iptables stop
chkconfig iptables off
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
setenforce 0

创建用户

useradd why
echo "**********" |passwd --stdin why

ssh端口修改,禁止root用户登录,使用新的用户登录,并给予新用户sudo权限

主机安全

阿里云安全组配置禁ping,禁22端口,使用新的ssh端口

升级内核

rpm -ivh http://www.elrepo.org/elrepo-release-6-8.el6.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel install kernel-lt -y
sed -i s/default=1/default=0/ /etc/grub.conf
reboot

此次8月11号购买的阿里云会需要修改/boot/grub/grub.conf文件的中default值为0

内核优化

配置环境

echo '* - nofile 65535' >> /etc/security/limits.conf
echo "alias vi='vim -r'" >> /etc/profile
echo "alias grep='grep --color'" >> /etc/profile
. /etc/profile

安装软件包

yum install -y lrzsz
yum install -y mysql-server mysql 
yum install -y git
yum install -y bind-utils

其他操作

  • 安装python2.7和python3.5,并创建其虚拟环境
yum install zlib zlib-devel openssl  openssl-devel readline readline-devel

启动docker

下载docker并下载CentOS镜像

yum install -y docker-io
service docker start
chkconfig docker on
chkconfig docker --list
docker pull centos

启动镜像

docker run --name web -it centos
yum install -y util-linux
# 需要通过nsenter进入docker容器
docker inspect --format "{{.State.Pid}}" web
# 获取容器pid2089
nsenter --target `docker inspect --format "{{.State.Pid}}" web` --mount --uts --ipc --net --pid

docker环境准备

openresty

# wget
yum install -y wget
# openresty依赖
yum install readline-devel pcre-devel openssl-devel gcc
# perl
yum install -y perl-devel perl-ExtUtils-Embed
# make
yum install -y make
# 下载openresty
wget https://openresty.org/download/openresty-1.11.2.3.tar.gz -P /usr/local/openresty
tar xf /usr/local/openresty/openresty-1.11.2.3.tar.gz  -C /opt/
cd /opt/openresty-1.11.2.3/
./configure --prefix=/opt/openresty --with-luajit --without-http_redis2_module --with-http_iconv_module --with-http_perl_module 
gmake
gmake install

python依赖包

安装pip

wget https://pypi.python.org/packages/07/a0/11d3d76df54b9701c0f7bf23ea9b00c61c5e14eb7962bb29aed866a5844e/setuptools-36.2.7.zip#md5=b9e6c049617bac0f9e908a41ab4a29ac
unzip setuptools-36.2.7.zip
yum install y unzip
unzip setuptools-36.2.7.zip
cd setuptools-36.2.7
python setup.py install
cd ..
wget "https://pypi.python.org/packages/source/p/pip/pip-1.5.4.tar.gz#md5=834b2904f92d46aaa333267fb1c922bb"
tar pip-1.5.4.tar.gz 
tar xf  pip-1.5.4.tar.gz 
cd pip-1.5.4
python setup.py install

下载依赖包

pip install django==1.6
pip install uwsgi
# 如果出现Python.h: No such file or directory,解决方式yum install -y python-devel
pip install markdown2
pip install MySQL-python
# 最好安装以下依赖

遇到问题

忘记挂载目录了

导出容器,导入进行

[root@why docker_fs]# docker stop c5f764263b5c
c5f764263b5c
[root@why docker_fs]# docker export !$ > web.docker
docker export c5f764263b5c > web.docker
[root@why docker_fs]# cat web.docker | docker import - web1.0
2f58e99fcd8643135f97d1d0a31f394f15e3eb4dc0532125fd6c8b4cc8da8d20
[root@why docker_fs]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
web1.0              latest              2f58e99fcd86        3 minutes ago       584.6 MB
centos              latest              f3b88ddaed16        8 days ago          192.5 MB

可以看到docker的容器导入的时候是以镜像的形式进行

CentOS7容器的一个坑

[root@web1 init.d]# systemctl start mariadb.service 
Failed to get D-Bus connection: Operation not permitted

解决方式

docker run --privileged --name webserver1.0 -p 80:80 -p 443:443 -v /opt/docker_fs/:/data -it web1.0 /usr/sbin/init

配置数据库

启动数据库

nsenter --target `docker inspect --format "{{.State.Pid}}" webserver1.0` --mount --uts --ipc --net --pid
yum install -y mariadb*
systemctl start mariadb.service

创建所需库

mysql
CREATE DATABASE why DEFAULT CHARSET=utf8;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES ON why.* TO 'why'@'localhost' IDENTIFIED BY '123456';

导入数据

[root@6865188c83d0 code]# mysql why < mysite_bak_2017_8_13.sql
[root@6865188c83d0 code]# mysql -e "select count(*) from why.blog_blogpost;"
+----------+
| count(*) |
+----------+
|      113 |
+----------+

安装ss

yum install -y iproute

启动服务

/opt/openresty/nginx/sbin/nginx
cd /data/code/blog/ && uwsgi --http :8000 --plugin python --wsgi-file mysite/wsgi.py >/dev/null 2>&1 &

整个过程网站迁移完成

启动docker的问题

top - 10:13:32 up 2 days, 19:00,  1 user,  load average: 1.00, 1.01, 1.05
Tasks: 104 total,   3 running, 101 sleeping,   0 stopped,   0 zombie
Cpu(s): 20.2%us, 79.8%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1020588k total,   909536k used,   111052k free,   124064k buffers
Swap:        0k total,        0k used,        0k free,   495520k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                                         
15504 root      20   0  9848  848  720 R 99.8  0.1   3440:09 agetty                                                                                                                                                                                                           
    1 root      20   0 19280 1260  984 S  0.0  0.1   0:00.48 init                                                                                                                                                                                                             
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.02 kthreadd                                                                                                                                                                                                         
    3 root      20   0     0    0    0 S  0.0  0.0   0:01.70 ksoftirqd/0                                                                                                                                                                                                      
    5 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0H                                                                                                                                                                                                     

出现cpu使用率为100%的问题

[root@why 10:19:50 ~]$ enterdocker 
[root@6865188c83d0 /]# systemctl stop getty@tty1.service
[root@6865188c83d0 /]# systemctl mask getty@tty1.service
Created symlink from /etc/systemd/system/getty@tty1.service to /dev/null.
[root@6865188c83d0 /]# systemctl status mariadb
● mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled)
   Active: active (running) since Sat 2017-08-12 16:49:06 UTC; 2 days ago
 Main PID: 222 (mysqld_safe)
   CGroup: /system.slice/mariadb.service
           ├─222 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
           └─378 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mariadb/mariadb.log --pid-file=/var/run/mariadb/mariadb.pid --socket=/var/lib/mysql/mysql.sock

Aug 12 16:49:03 6865188c83d0 mariadb-prepare-db-dir[146]: The latest information about MariaDB is available at http://mariadb.org/.
Aug 12 16:49:03 6865188c83d0 mariadb-prepare-db-dir[146]: You can find additional information about the MySQL part at:
Aug 12 16:49:03 6865188c83d0 mariadb-prepare-db-dir[146]: http://dev.mysql.com
Aug 12 16:49:03 6865188c83d0 mariadb-prepare-db-dir[146]: Support MariaDB development by buying support/new features from MariaDB
Aug 12 16:49:03 6865188c83d0 mariadb-prepare-db-dir[146]: Corporation Ab. You can contact us about this at sales@mariadb.com.
Aug 12 16:49:03 6865188c83d0 mariadb-prepare-db-dir[146]: Alternatively consider joining our community based development effort:
Aug 12 16:49:03 6865188c83d0 mariadb-prepare-db-dir[146]: http://mariadb.com/kb/en/contributing-to-the-mariadb-project/
Aug 12 16:49:03 6865188c83d0 mysqld_safe[222]: 170812 16:49:03 mysqld_safe Logging to '/var/log/mariadb/mariadb.log'.
Aug 12 16:49:03 6865188c83d0 mysqld_safe[222]: 170812 16:49:03 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Aug 12 16:49:06 6865188c83d0 systemd[1]: Started MariaDB database server.

[root@6865188c83d0 /]# logout
  • systemctl stop getty@tty1.service
  • systemctl mask getty@tty1.service 上边的两个操作分别为停止和屏蔽(也可以说是注销)

引起这个问题的原因是在使用"docker run"运行容器时使用了 "/sbin/init"和"--privileged"参数。

使用/sbin/init启动容器并加上--privileged参数,相当于docker容器获得了宿主机的全权委托权限。这时docker容器内部的init与宿主机的init产生了混淆。

为了解决这个问题,docker后来的版本中docker run增加了两个选项参数"--cap-add"和"--cap-drop"。

  • --cap-add : 获取default之外的linux的权限
  • --cap-drop: 放弃default linux权限

参考博客

所以,在运行容器时,可以不用--privileged参数的尽量不用,用--cap-add参数替代。如果必须使用--privileged=true参数的,可以通过在宿主机和容器中执行以下命令将agetty关闭。

top - 11:14:31 up 2 days, 20:01,  3 users,  load average: 0.00, 0.01, 0.05
Tasks: 115 total,   1 running, 114 sleeping,   0 stopped,   0 zombie
Cpu(s): 18.5%us, 66.3%sy,  0.0%ni, 15.1%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1020588k total,   922560k used,    98028k free,   125212k buffers
Swap:        0k total,        0k used,        0k free,   500580k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                                         
    1 root      20   0 19280 1260  984 S  0.0  0.1   0:00.48 init                                                                                                                                                                                                             
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.02 kthreadd                                                                                                                                                                                                         
    3 root      20   0     0    0    0 S  0.0  0.0   0:01.70 ksoftirqd/0                                                                                                                                                                                                      
    5 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0H                                                                                                                                                                                                     
    7 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0                                

cpu使用率恢复正常。+

定时任务

yum install crontabs
start crond.service
status crond.service
● crond.service - Command Scheduler
   Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2017-08-25 18:53:03 CST; 6s ago
 Main PID: 4721 (crond)
   CGroup: /system.slice/crond.service
           └─4721 /usr/sbin/crond -n

Aug 25 18:53:03 6865188c83d0 systemd[1]: Started Command Scheduler.
Aug 25 18:53:03 6865188c83d0 systemd[1]: Starting Command Scheduler...
Aug 25 18:53:03 6865188c83d0 crond[4721]: (CRON) INFO (Syslog will be used instead of sendmail.)
Aug 25 18:53:03 6865188c83d0 crond[4721]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 62% if used.)
Aug 25 18:53:03 6865188c83d0 crond[4721]: (CRON) INFO (running with inotify support)

supervisor管理进程

首先使用gunicorn启动web服务,django文档参考

官方关于gunicorn文档

某度了半小时没一个讲的清楚的。

[root@6865188c83d0 /]# pip install gunicorn
[root@6865188c83d0 /]# cd /data/code/blog/
[root@6865188c83d0 blog]# tree ./
./
├── blog
│   ├── admin.py
│   ├── admin.pyc
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── models.py
│   ├── models.pyc
│   ├── templatetags
│   │   ├── custom_markdown.py
│   │   ├── custom_markdown.pyc
│   │   ├── custom_markdown.py.old
│   │   ├── __init__.py
│   │   └── __init__.pyc
│   ├── tests.py
│   ├── urls.py
│   ├── urls.pyc
│   ├── views.py
│   └── views.pyc
├── DEVELOP.md
├── manage.py
├── manage.pyc
├── mysite
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── settings.py
│   ├── settings.pyc
│   ├── urls.py
│   ├── urls.pyc
│   ├── uwsgi.ini
│   ├── views.py
│   ├── views.pyc
│   ├── wsgi.py
│   └── wsgi.pyc
└── templates
    ├── bak
    │   ├── blog_base.html.old.green
    │   └── blog_base.html.old.yahu
    ├── blog_base.html
    ├── blog_every.html
    ├── blog_every_tag.html
    ├── blog.html
    ├── blog_post.html
    ├── blog_tag.html
    ├── index.html.old
    ├── index.html.willnew
    ├── jianli.html
    ├── login.html
    ├── logout.html
    ├── me.html
    └── search.html

[root@6865188c83d0 blog]# gunicorn mysite.wsgi -b 127.0.0.1:8001
[2017-09-26 17:27:04 +0000] [23093] [INFO] Starting gunicorn 19.7.1
[2017-09-26 17:27:04 +0000] [23093] [INFO] Listening at: http://127.0.0.1:8001 (23093)
[2017-09-26 17:27:04 +0000] [23093] [INFO] Using worker: sync
[2017-09-26 17:27:04 +0000] [23098] [INFO] Booting worker with pid: 23098



^C[2017-09-26 17:33:04 +0000] [23093] [INFO] Handling signal: int
[2017-09-26 17:33:04 +0000] [23098] [INFO] Worker exiting (pid: 23098)
[2017-09-26 17:33:04 +0000] [23093] [INFO] Shutting down: Master

supervisor使用

[root@6865188c83d0 /]# pip install supervisor
Downloading/unpacking supervisor
  Downloading supervisor-3.3.3.tar.gz (418kB): 418kB downloaded
  Running setup.py (path:/tmp/pip_build_root/supervisor/setup.py) egg_info for package supervisor

    warning: no previously-included files matching '*' found under directory 'docs/.build'
Downloading/unpacking meld3>=0.6.5 (from supervisor)
  Downloading meld3-1.0.2-py2.py3-none-any.whl
Installing collected packages: supervisor, meld3
  Running setup.py install for supervisor

    warning: no previously-included files matching '*' found under directory 'docs/.build'
    Skipping installation of /usr/lib/python2.7/site-packages/supervisor/__init__.py (namespace package)
    Installing /usr/lib/python2.7/site-packages/supervisor-3.3.3-py2.7-nspkg.pth
    Installing echo_supervisord_conf script to /usr/bin
    Installing pidproxy script to /usr/bin
    Installing supervisorctl script to /usr/bin
    Installing supervisord script to /usr/bin
Successfully installed supervisor meld3
Cleaning up...
[root@6865188c83d0 /]# echo_supervisord_conf > /etc/supervisord.conf
[root@6865188c83d0 /]# vi /etc/supervisord.conf 
;[include]
;files = relative/directory/*.ini
改为
[include]
files = /etc/supervisor/conf.d/*.ini
[root@6865188c83d0 /]# mkdir /etc/supervisor/conf.d/ -p
[root@6865188c83d0 /]# vi /etc/supervisor/conf.d/blog.ini
[program:blog]
command = gunicorn mysite.wsgi -b 127.0.0.1:8000
directory = /data/code/blog/
autostart = true
autorestart = true
user=root
startretries = 3
startsecs = 5
redirect_stderr = true
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /data/log/supervisor/blog_stdout.log
[root@6865188c83d0 /]# supervisord -c /etc/supervisord.conf
[root@6865188c83d0 blog]# supervisorctl status blog
blog                             RUNNING   pid 23183, uptime 0:00:12
[root@6865188c83d0 blog]# ss -nlpt | grep 8000
LISTEN     0      128    127.0.0.1:8000                     *:*                   users:(("gunicorn",pid=23188,fd=5),("gunicorn",pid=23183,fd=5))

我uwsgi的话,supervisor启动的时候会报No such file or directory [core/utils.c line 3686],暂时还没有解决。

以上问题已解决,配置文件参考

[program:blog]
command = uwsgi --http :8000 --wsgi-file mysite/wsgi.py
directory = /data/code/blog/
autostart = true
autorestart = true
user=root
startretries = 3
startsecs = 5
redirect_stderr = true
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /data/log/supervisor/blog2_stdout.log

未完待续

七牛云加速

七牛链接

创建用户

进行个人注册

填写相关信息

登录

主页面

实名认证

绑定支付宝

重新登录

对象存储

新建存储空间

配置镜像源,以下流程我还没有跑通

融合CDN

融合CDN虽然要给免费的额度,但是需要充值10块。

新建融合CDN

新建融合CDN配置

七牛自动进行七牛域名CNAME到回源域名

配置回源域名

DNS对应配置

配置成功

同步存储空间

当然我没用这种方式

[root@why opt]# cd tools/
[root@why tools]# wget http://devtools.qiniu.com/linux/amd64/qrsctl-v3.2.2017050
[root@why tools]# ll
total 8368
-rwxr--r-- 1 root root 8568110 May  1 23:20 qrsctl-v3.2.20170501
[root@why tools]# ./qrsctl-v3.2.20170501 login 123456@163.com 123456
[root@why tools]# ./qrsctl-v3.2.20170501 info
UserId:      123456@163.com
Uid:         1381194907
Email:       123456@163.com
UserType:    stduser(0x4)
DeviceNum:   0
InvitationNum:   0
[root@why tools]# ./qrsctl-v3.2.20170501 buckets
[static]
[root@why tools]# ./qrsctl-v3.2.20170501 img static http://image.whysdomain.com http://image.whysdomain.com

更多参考https://developer.qiniu.com/kodo/tools/1300/qrsctl