正文

docker Dockerfile 创建镜像

IP云
IP云V管理员/7 阅读
08/04
提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

Docker 组件

1. docker client ： docker的客户端

2. docker server : docker daemon的主要组成部分，接受用户通过docker client发送的请求，并按照响应的路由规则实时路由分发。

3. docker image ： docker镜像运行之后变成容器 （docker run）,启动快，采用了分层模式。

4.docker Registry： registry是 docker镜像的中央存储仓库(pull/push)

docker 使用yum 安装最新版

[root@docker1 yum.repos.d]# cat docker.repo
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/7
enabled=
gpgcheck=
gpgkey=https://yum.dockerproject.org/gpg

docker 安装

[root@docker1 ~]# yum -y install docker-engine启动docker
[root@docker1 ~]# systemctl start docker.service
[root@docker1 ~]# systemctl enable docker.service

使用dockerfile 生成docker镜像, 从docker.io 下载镜像。

搜索centos镜像
[root@docker1 ~]#docker search centospull centos镜像
[root@docker1 ~]#docker pull centos查看下载的镜像
[root@docker1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZEcentos latest 67591570dd29 weeks ago MB

[root@docker1 ~]# git clone https://git.oschina.net/dockerf/docker-training.git
[root@docker1 ~]# ls
docker-training

[root@docker1 ~]# cd docker-training
[root@docker1 docker-training]# ls

centos7 mysql php-fpm wordpress 4个目录
构建一个 centos7 php-fpm mysql wordpress 的docker镜像

[root@docker1 centos7]# ls
1.repo Centos-7.repo Dockerfile supervisord.conf

Dockerfile 是一个自动构建docker镜像的配置文件，

[root@docker1 centos7]# cat Dockerfile
#需要一个基础镜像，centos7.1.1503 是从docker上pull下来
#FROM centos:centos7.1.1503

FROM centos:latest

 

#维护者
MAINTAINER fengjian <fengjian@senyint.com>

#设置一个时区的环境变量
ENV TZ "Asia/Shanghai"

#虚拟终端
ENV TERM xterm

#dockerfile中有2条命令可以复制文件，1.copy 2.add， add比copy多2个功能，可以写成连接 直接copy到container，如果是压缩文件，add能自动解压
ADD Centos-.repo /etc/yum.repos.d/CentOS-Base.repo
ADD .repo /etc/yum.repos.d/epel.repoRUN yum install -y curl wget tar bzip2 libtool-ltdl-devel unzip vim-enhanced passwd sudo yum-utils hostname net-tools rsync man && \ yum install -y gcc gcc-c++ git make automake cmake patch logrotate python-devel libpng-devel libjpeg-devel && \ yum install -y python-pip #RUN pip install --upgrade pip

#supervisor 进程管理工具，运行单个进程可以不使用
RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple supervisor
ADD supervisord.conf /etc/supervisord.conf

#/etc/supervisor.conf.d 存放启动进程的配置文件
RUN mkdir -p /etc/supervisor.conf.d && \
mkdir -p /var/log/supervisor

#container想暴露22端口给宿主机

EXPOSE 22
#最后一条ENTRYPOINT 才能生效
ENTRYPOINT ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]

[root@docker1 centos7]# cat 1.repo
[bash]
name=centos7
baseurl=http://192.168.20.220/centos7/Packages/
enabled=1
gpgcheck=0

[root@docker1 centos7]# cat Centos-.repo
# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client. You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the
# remarked out baseurl= line instead.
#
#[base]
name=CentOS-$releasever - Base - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/os/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/os/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
gpgcheck=
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#released updates
[updates]
name=CentOS-$releasever - Updates - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/updates/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/updates/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
gpgcheck=
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/extras/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/extras/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
gpgcheck=
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/centosplus/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/centosplus/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
gpgcheck=
enabled=
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#contrib - packages by Centos Users
[contrib]
name=CentOS-$releasever - Contrib - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/contrib/$basearch/
http://mirrors.aliyuncs.com/centos/$releasever/contrib/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contrib
gpgcheck=
enabled=
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7

[root@docker1 centos7]# cat supervisord.conf
[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod= ; socket file mode (default )[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB
logfile_backup=
loglevel=info
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=true ; (Start in foreground if true; default false)
minfds= ; (min. avail startup file descriptors;default )
minprocs= ; (min. avail process descriptors;default )[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket[include]
files = /etc/supervisor.conf.d/*.conf

从docker.io 下载centos,根据dockerfile 生成docker镜像, 
[root@docker1 centos7]# docker build -t fengjian/centos:7.3  .

查看生成的镜像

[root@docker1 centos7]#  docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
fengjian/centos 7.1 03a49ca4b7b9 days ago 667.8 MB

通过docker镜像 生成docker 容器。 docker run命令

-p 小写， container 端口，指定到宿主机的端口

-P  大写， container的 端口，映射到宿主机的 随机端口

生成docker 容器

[root@docker1 ~]#docker run -d -p 2222:22 --name base fengjian/centos:7.3

查看容器信息

[root@docker1 ~]#docker ps -a

[root@docker1 ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ebce60d09d31 fengjian/centos:7.1 "/usr/bin/supervisord" About a minute ago Up About a minute /tcp, 0.0.0.0:->/tcp base

进入docker container中
[root@docker1 ~]# docker exec -it ebce60d09d31 /bin/bash

制作php-fpm的镜像

[root@docker1 php-fpm]# ls
Dockerfile nginx_default.conf nginx_nginx.conf php_www.conf supervisor_nginx.conf supervisor_php-fpm.conf

[root@docker1 php-fpm]# vim Dockerfile FROM fengjian/centos:7.3
MAINTAINER fengjian <fengjian@senyint.com># Set environment variable
ENV APP_DIR /appRUN yum -y swap -- remove fakesystemd -- install systemd systemd-libs && \
yum -y install nginx php-cli php-mysql php-pear php-ldap php-mbstring php-soap php-dom php-gd php-xmlrpc php-fpm php-mcrypt && \
yum clean allADD nginx_nginx.conf /etc/nginx/nginx.conf
ADD nginx_default.conf /etc/nginx/conf.d/default.confADD php_www.conf /etc/php-fpm.d/www.conf
RUN sed -i 's/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/' /etc/php.iniRUN mkdir -p /app && echo "<?php phpinfo(); ?>" > ${APP_DIR}/info.phpEXPOSE ADD supervisor_nginx.conf /etc/supervisor.conf.d/nginx.conf
ADD supervisor_php-fpm.conf /etc/supervisor.conf.d/php-fpm.confONBUILD ADD . /app
ONBUILD RUN chown -R nginx:nginx /app

[root@docker1 php-fpm]# vim supervisor_nginx.conf [program:nginx]
directory=/
command=/usr/sbin/nginx -c /etc/nginx/nginx.conf
user=root
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log[program:php-fpm]
directory=/
command=/usr/sbin/php-fpm
user=root
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log

创建php-fpm的 镜像

[root@docker1 php-fpm]# docker build -t fengjian/php-fpm:5.4  .

启动一个container

[root@docker1 php-fpm]# docker run -d -p 8080:80 --name wesite fengjian/php-fpm:5.4

访问宿主机的 8080端口

http://192.168.20.209:8080/info.php

创建mysql镜像

[root@docker1 mysql]# docker build -t fengjian/mysql:5.5 .

###docker run -d -p 3306:3306 -v host_dir:container_dir   -v参数是映射container路径到宿主机上###########

[root@docker1 mysql]# docker run -d -p 3306:3306 -v  /data/mysql/data:/var/lib/mysql  --name dbserver fengjian/mysql:5.5

宿主机上的/data/mysql/data目录文件

[root@docker1 data]# ll /data/mysql/data/
total
-rw-rw---- Jan : aria_log.
-rw-rw---- Jan : aria_log_control
drwx------ Jan : fengjian
-rw-rw---- Jan : ibdata1
-rw-rw---- Jan : ib_logfile0
-rw-rw---- Jan : ib_logfile1
drwx------ root Jan : mysql
srwxrwxrwx Jan : mysql.sock
drwx------ Jan : performance_schema
drwx------ root Jan : test
[root@docker1 data]#

删除container后， 在生成新的container 数据还可以继续使用。
[root@docker1 data]#  docker rm -f dbserver (删除)
[root@docker1 data]#  docker run -d -p 3306:3306 -v /data/mysql/data:/var/lib/mysql --name newmysqldb  fengjian/mysql:5.5

构建动态网站wordpress, 使用php-fpm镜像

[root@docker1 wordpress]# ls
Dockerfile init.sh readme.html wp-admin wp-comments-post.php wp-content wp-includes wp-load.php wp-mail.php wp-signup.php xmlrpc.php
index.php license.txt wp-activate.php wp-blog-header.php wp-config-sample.php wp-cron.php wp-links-opml.php wp-login.php wp-settings.php wp-trackback.php

[root@docker1 wordpress]# vim Dockerfile

from fengjian/php-fpm:5.4add init.sh /init.shentrypoint ["/init.sh", "/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]
#先执行 /init.sh脚本， 然后再执行后面的服务，其实是启动了nginx 和php-fpm

父镜像php-fpm 的Dockerfile最后两行

ONBUILD ADD . /app
ONBUILD RUN chown -R nginx:nginx /app

ONBUILD 在构建 wordpress 的时候生效

所有的代码文件全部copy到/app下， 但是Dockerfile没有作用，所以 可以在目录下新建一个.dockerignore， 排除Dockerfile

[root@docker1 wordpress]# vim .dockerignore

Dockerfile

创建wordpress镜像，版本4.2

[root@docker1 wordpress]# docker build -t fengjian/wordpress:4.2  .

查看镜像

[root@docker1 wordpress]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
fengjian/wordpress 4.2 8591f07cc2e2 seconds ago 848.4 MB
fengjian/mysql 5.5 b54f78aeefb8 hours ago 848.2 MB
fengjian/php-fpm 5.4 fc1856e25486 hours ago 810.8 MB
fengjian/centos 7.1 fbafb1b36c30 hours ago 712.8 MB
tomcat latest 47bd812c12f6 weeks ago 355.2 MB
mysql latest 594dc21de8de weeks ago 400.1 MB
centos centos7.1.1503 285396d0a019 months ago 212.1 MB
centos centos7.1.1503 285396d0a019 months ago 212.1 MB
kubeguide/tomcat-app v1 a29e200a18e9 months ago 358.2 MB

启动一个容器， 使用-e 参数，传递环境变量,WORDPRESS_DB_HOST是在init.sh 脚本中定义的

[root@docker1 wordpress]# docker run -d -p : --name wordpress -e WORDPRESS_DB_HOST=192.168.20.209 -e WORDPRESS_DB_USER=fengjian -e WORDPRESS_DB_PASSWORD= fengjian/wordpress:4.2

访问ip的80端口

ENTRYPOINT 与 CMD 命令区别运行一个Docker容器像运行一个程序一样,如果写了10条，只有最后一条生效。. ENTRYPOINT ["executable","param1","param2"]
. ENTRPOINT command param1 param2 (shell from)docker run -it -entrypoint=覆盖 Dockerfile ENTRYPOINT[]CMD 用法
1.CMD["executable","param1","param2"] （exec from , this is the preferred form）
第一种用法:运行一个可执行的文件并提供参数2.CMD [“param1”,"param2"] （as default parameters to ENTRYPOINT）
第二种用法： 为ENTRPOINT指定参数3.CMD command param1 param2 （sehll form）
第三种用法(shell form)： 是以"/bin/sh -c" 的方法执行的命令例子：
CMD ["/bin/echo","This is test CMD"]
docker run -it -rm fengjian/cmd:0.1 /bin/bash

#################################################################################################

第二docker 实战之 Registry 以及持续集成

构建一个企业内部的registry

docker1: 192.168.20.209

registry: 192.168.20.135  5000端口

[root@registry ~]# docker search registry

再次把registry pull到本地

[root@registry ~]# docker pull registry
Using default tag: latest
latest: Pulling from library/registry
b7f33cc0b48e: Pull complete
46730e1e05c9: Pull complete
: Pull complete
0cf045fea0fd: Pull complete
b78a03aa98b7: Pull complete
Digest: sha256:0e40793ad06ac099ba63b5a8fae7a83288e64b50fe2eafa2b59741de85fd3b97
Status: Downloaded newer image for registry:latest

查看docker1 192.168.20.209  镜像

[root@docker1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
fengjian/wordpress 4.2 8591f07cc2e2 days ago 848.4 MB
fengjian/mysql 5.5 b54f78aeefb8 days ago 848.2 MB
fengjian/php-fpm 5.4 fc1856e25486 days ago 810.8 MB
fengjian/centos 7.1 fbafb1b36c30 days ago 712.8 MB
registry latest d1e32b95d8e8 days ago 33.17 MB
tomcat latest 47bd812c12f6 weeks ago 355.2 MB
mysql latest 594dc21de8de weeks ago 400.1 MB
centos centos7.1.1503 285396d0a019 months ago 212.1 MB
centos centos7.1.1503 285396d0a019 months ago 212.1 MB

查看registry 192.168.20.135  镜像

[root@registry ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry latest d1e32b95d8e8 weeks ago 33.2 MB

registry启动一个 registry的容器

[root@registry ~]# docker run -d -p :5000 -v /opt/registry:/var/lib/registry -p 5000:5000 --restart=always --name registry registry:latest

Registry服务默认会将上传的镜像保存在容器的/var/lib/registry，我们将主机的/opt/registry目录挂载到该目录，即可实现将镜像保存到主机的/opt/registry目录了。

registry_url: 启动的registry，本地registry:5000地址

namespace ： 指定目录

name： 镜像的名字

registry_url/namespace/tomcat:v1.0

通过 docker tag 可以为容器打一个标记，类似于别名的作用，

[root@docker1 ~] docker pull fengjian/fengjian

对fengjian 打标记成 192.168.20.209:5000/fengjian/fengjian/fengjian:20170122v1.0

[root@docker1 ~]# docker tag fengjian/fengjian:latest 192.168.20.135:5000/fengjian/fengjian/fengjian:20170122v1.0

查看镜像

[root@docker1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
fengjian/wordpress 4.2 8591f07cc2e2 2 days ago 848.4 MB
fengjian/mysql 5.5 b54f78aeefb8 3 days ago 848.2 MB
fengjian/php-fpm 5.4 fc1856e25486 3 days ago 810.8 MB
fengjian/centos 7.1 fbafb1b36c30 3 days ago 712.8 MB
registry latest d1e32b95d8e8 4 days ago 33.17 MB
tomcat latest 47bd812c12f6 5 weeks ago 355.2 MB
mysql latest 594dc21de8de 5 weeks ago 400.1 MB
centos centos7.1.1503 285396d0a019 4 months ago 212.1 MB
centos centos7.1.1503 285396d0a019 4 months ago 212.1 MB
kubeguide/tomcat-app v1 a29e200a18e9 6 months ago 358.2 MB
192.168.20.135:5000/fengjian/fengjian/fengjian 20170122v1.0 3468c34fa83b 13 months ago 97.95 MB
fengjian/fengjian latest 3468c34fa83b 13 months ago 97.95 MB运行docker push将hello-world镜像push到我们的私有仓库中

[root@docker ~]# docker push  192.168.20.135:5000/fengjian/fengjian/fengjian:20170122v1.0

The push refers to a repository [192.168.20.135:5000/fengjian/nginx20170203]
  Get https://192.168.20.135:5000/v1/_ping: http: server gave HTTP response to HTTPS client

出现无法push镜像到私有仓库的问题。这是因为我们启动的registry服务不是安全可信赖的。这是我们需要修改docker的配置文件/usr/lib/systemd/system/docker.service，添加下面的内容，注意： registry 和 docker1 2台服务器都需要修改

上传镜像到 192.168.20.135

[root@docker overlay]# docker push  192.168.20.135:5000/fengjian/fengjian/fengjian:20170122v1.0

The push refers to a repository [192.168.20.135:5000/fengjian/fengjian/fengjian:20170122v1.0]

23c8d40ebb9e: Pushed
5526182de2ab: Pushed
652f3c2c3f57: Pushed
bf76891beffc: Pushed
f696adb3bd45: Pushed
46db44806cd4: Pushed
2dd577fe2559: Pushed
bbc4847eb1d2: Pushed
747f5baee8ac: Pushed
29003cbb49e1: Pushed
f5d4b5d6f2ff: Pushed
ee745a500b91: Pushed
3383431a5cc0: Pushed
8aabcc6c5e8d: Pushed
967105df7f61: Pushed
0c051da11cb4: Pushed
34e7b85d83e4: Pushed
v1: digest: sha256:4e5d763dfb99ecd95128d1033e14bb4740613045c89bb2646006ac7db08f5a6f size: 3871

通过浏览器，查询上传结果

使用docker pull从我们的私有仓库中获取192.168.20.135:5000/fengjian/fengjian/fengjian:20170122v1.0镜像

[root@docker ~ ]# docker pull 192.168.20.135:5000/fengjian/nginx20170203:v1

v1: Pulling from fengjian/nginx20170203
17385548ba54: Already exists
59da822a5404: Already exists
ec5de50f3658: Already exists
751fb563feef: Already exists
8145f1a2090b: Already exists
575600a5843d: Already exists
035deb98f67f: Already exists
2e1f8c7e36ce: Already exists
3cf27705cd77: Pull complete
d4e37a9633b1: Pull complete
1aab1e953ef2: Pull complete
31afde0ced92: Pull complete
253eadce8153: Pull complete
750606d876c8: Pull complete
f96cc19c204f: Pull complete
eea9946ffb66: Pull complete
da59d6a4a8bd: Pull complete
Digest: sha256:4e5d763dfb99ecd95128d1033e14bb4740613045c89bb2646006ac7db08f5a6f
Status: Downloaded newer image for 192.168.20.135:5000/fengjian/nginx20170203:v1

192.168.20.209启动一个容器

[root@docker ~ ]# docker run -d -p 8081:80 --name nginx 192.168.20.135:5000/fengjian/nginx20170203:v1

[root@docker ~ ]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7539cd87c9bd 192.168.20.135:5000/fengjian/nginx20170203:v1 "/usr/bin/supervis..." 2 minutes ago Up 2 minutes 22/tcp, 0.0.0.0:8081->80/tcp nginx

登陆到容器中

[root@docker overlay]# docker exec -it nginx /bin/bash

容器nginx启动正常

[root@7539cd87c9bd nginx-1.11.2]# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 17:21 ? 00:00:00 /usr/bin/python2 /usr/bin/supervisord -n -c /etc/supervisord.conf
root 9 1 0 17:21 ? 00:00:00 nginx: master process /data/nginx/sbin/nginx
nobody 10 9 0 17:21 ? 00:00:00 nginx: worker process
root 88 0 1 17:25 ? 00:00:00 /bin/bash
root 104 1 0 17:25 ? 00:00:00 /data/nginx/sbin/nginx
root 105 88 0 17:25 ? 00:00:00 ps -ef

注： registry服务器，暂时看不到上传的image镜像。已经存在的镜像，在

[root@registry repositories]# pwd
/data/registry/docker/registry/v2/repositorie

docker registry https服务

1. 启动registry仓库的镜像，与上面相同
[root@registry ~]# docker run -d -p 5000:5000 -v /opt/registry:/var/lib/registry --restart=always --name registry registry:latest

. 关闭 /usr/lib/systemd/system/docker.service 配置文件中 "--insecure-registry=192.168.20.135:5000"
[root@registry ~]# systemctl daemon-reload
[root@registry ~]# systemctl restart docker.service. 启动 nginx容器 映射443端口
[root@registry ~]# docker pull nginx (也可以自己制作nginx镜像)
运行nginx容器
[root@registry ~]# docker run -d -p : --name nginx nginx:latest4. 修改nginx 配置文件，已经添加域名证书
[root@registry ~]# docker cp nginx.conf /etc/nginx/nginx.conf
[root@registry ~]# docker cp sslkey /etc/nginx/

5. 登陆到容器中重新启动nginx
[root@123131nginx ~]# /etc/init.d/nginx restart########################
nginx.conf 配置文件events {
worker_connections ;
}http { upstream docker-registry {
server 192.168.20.135:;
} server {
listen ssl;
server_name docker.cinyi.com; ssl_certificate /data/nginx/sslkey/cinyi.crt;
ssl_certificate_key /data/nginx/sslkey/cinyi.key; client_max_body_size ; chunked_transfer_encoding on; location / {
proxy_pass http://docker-registry;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout ;
}
}
}6. 在 192.168.20.209 服务器上，查看镜像，对镜像打tag，然后push 到registry[root@docker1 ~]# docker images

[root@docker1 ~]# docker tag senyint/im-web docker.cinyi.com:443/senyint/im-web:443

[root@docker1 ~]# docker push  docker.cinyi.com:443/senyint/im-web:443

 

查看registry 里面存储的镜像使用 Registry V2 API。可以列出所有镜像：curl http://<私有registry地址>/v2/_catalog例如
[root@docker225 ~]# curl https://docker.cinyi.com/v2/_catalog
{"repositories":["fengjian/nginx20170203","mysql20170203","senyint/centos7.3","senyint/im-web","senyint/nginx"]}

查看registry 里面存储的镜像tag使用 Registry V2 API。可以列出所有镜像：curl GET <protocol>://<registry_host>/v2/<镜像名>/tags/list例如
[root@docker225 ~]# curl GET https://docker.cinyi.com/v2/senyint/im-web/tags/list
{"name":"senyint/im-web","tags":["latest",""]}

删除docker.cinyi.com docker registry的镜像

1.在启动仓库时，需在配置文件中的storage配置中增加delete=true配置项，允许删除镜像
[root@registry ~]#tail -f /etc/docker/registry/config.yml文件 ,修改容器yml文件后，一直显示在报错，后续在处理

storage:
delete:
enabled: true

. 拿到disgest_hash参数curl --header "Accept: application/vnd.docker.distribution.manifest.v2+json" -I -X GET https://docker.cinyi.com/v2/senyint/nginx/manifests/latestHTTP/1.1 OK
Server: nginx/1.11.
Date: Wed, Feb :: GMT
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Content-Length:
Connection: keep-alive
Docker-Content-Digest: sha256:609a595020f0827301064ebc07b3ec3a5751641ef975a7a186518cf6b0d70f63
Docker-Distribution-Api-Version: registry/2.0
Etag: "sha256:609a595020f0827301064ebc07b3ec3a5751641ef975a7a186518cf6b0d70f63"
X-Content-Type-Options: nosniff.复制disgest_hash
Docker-Content-Digest: <digest_hash>

Docker-Content-Digest: sha256:609a595020f0827301064ebc07b3ec3a5751641ef975a7a186518cf6b0d70f63

.删除registry镜像

curl -I -X DELETE <protocol>://<registry_host>/v2/<repo_name>/manifests/<digest_hash>

[root@docker225 ~]# curl -I  -X DELETE https://docker.cinyi.com/v2/senyint/im-web/manifests/sha256:609a595020f0827301064ebc07b3ec3a5751641ef975a7a186518cf6b0d70f63第二种方法1. 打开镜像的存储目录， 删除镜像文件夹

[root@registry repositories]# docker exec registry rm -rf senyint

2.执行垃圾回收操作
  [root@registry repositories]# docker exec registry /bin/registry garbage-collect /etc/docker/registry/config.yml

3. 重启容器

[root@registry repositories]#  docker restart registry

docker-compose编排工具安装，一次启动多个容器。

#对安装好的pip进行一次升级
pip install --upgrade pip安装docker-compose
pip install docker-compose运行docker-compose
出现报错
pkg_resources.DistributionNotFound: backports.ssl-match-hostname>=3.5使用pip 更新backports.ssl-match-hostname的版本
pip install --upgrade backports.ssl_match_hostname
更新backports.ssl_match_hostname 到3.5版本后问题解决

[root@docker1 certs]# docker-compose up
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?

Supported filenames: docker-compose.yml, docker-compose.yaml

编写docker-compose.yml 文件

[root@docker1 second]# vim docker-compose.yml

mysql:
image: fengjian/mysql:5.5
ports:
- "3306:3306"
volumes:
- /var/lib/docker/vfs/dir/dataxc:/var/lib/mysql
hostname: mydb.server.com

tomcat:
image: tomcat
ports:
- "8080:8080"
links:
- mysql:db
environment:
- TOMCAT_USER=admin
- TOMCAT_PASS=admin
hostname: tomcat.server.com

在后台启动，在docker-compose.yml 下启动

[root@docker1 second]# docker-compose up -d

查看启动的容器

[root@docker1 second]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5b844baf351e tomcat "catalina.sh run" 6 minutes ago Up 6 minutes 0.0.0.0:8080->8080/tcp second_tomcat_1
f88ccf720119 fengjian/mysql:5.5 "/scripts/start" 6 minutes ago Up 6 minutes 22/tcp, 0.0.0.0:3306->3306/tcp second_mysql_1

停掉2个container

[root@docker1 second]# docker-compose stop

通过ps 查看通过docker-compose启动的有那些container。

[root@docker1 second]# docker-compose ps
Name Command State Ports
----------------------------------------------------
second_mysql_1 /scripts/start Exit 137
second_tomcat_1 catalina.sh run Exit 143

删除通过docker-compose建立的2个container.

[root@docker1 second]# docker-compose rm
Name Command State Ports
----------------------------------------------------
second_mysql_1 /scripts/start Exit 137
second_tomcat_1 catalina.sh run Exit 143

 

通过docker仓库自动构建（jenkins）

构建jenkins 镜像
构建maven镜像：

[root@docker /]# mkdir maven-tar

[root@docker /]# cd maven-tar/

[root@docker maven-tar]# wget http://mirror.bit.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz

docker最大的优势在于部署，jenkins最强大的在于作业调度和插件系统，如何结合两者？

创建一个jenkins镜像

apache-maven-3.3.9-bin.tar.gz  Dockerfile  jdk.tar.gz  jenkins.war  rc.local  settings.xml  supervisor_tomcat.conf  tomcatDockerFile 文件

FROM centos7.:
MAINTAINER fengjian <fengjian@senyint.com># Install maven
ADD apache-maven-3.3.-bin.tar.gz /data/
ADD jdk.tar.gz /data/
COPY tomcat /data/tomcat
COPY jenkins.war /data/tomcat/webapps/COPY settings.xml /data/maven/conf/settings.xml
ADD supervisor_tomcat.conf /etc/supervisor.conf.d/tomcat.conf

supervisor tomcat的启动配置文件

[root@docker maven-tar]# vim supervisor_tomcat.conf [program:tomcat]
directory=/
command=/data/tomcat/bin/catalina.sh start
user=root
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log
~

tomcat 启动文件 vim /data/tomcat/bin/catalina.sh

export JENKINS_HOME="/data/jenkins_home"
export JAVA_HOME=/data/jdk
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib# OS specific support. $var _must_ be set to either true or false.

创建jenkins镜像， 启动jenkins镜像

[root@docker ~]# docker build -t jenkins .
[root@docker ~]# docker run -d -p : --name jenkins jenkins

把 container 容器 打包成镜像

. 关闭jenkins
[root@docker ~ ]# docker stop jenkinsroot@docker maven-tar]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9174cf36cdfc jenkins "/usr/bin/supervis..." hours ago Up About an hour /tcp, 0.0.0.0:->/tcp jenkins. 把容器打包成镜像
[root@docker ~ ]# docker commit 9174cf36cdfc jenkins201702043. 查看新的jenkins 镜像
[root@docker ~] docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
jenkins20170204 latest 5254a69cb614 41 seconds ago 1.62 GB

jenkins镜像里内置了docker client命令行工具，/usr/bin/docker，因此我们只需要传递 DOCKER_HOST 环境变量 或者映射 docker.sock 文件给jenkins容器，就可以让jenkins容器里面拥有docker的操作能力，进而将两者结合起来。创建jenkins 新的镜像，把/usr/bin/docker 和 /var/run/docker.sock 映射给 jenkins 容器

创建jenkins 新的镜像，把/usr/bin/docker 和 /var/run/docker.sock 映射给 jenkins 容器

docker run -d -p : -v /usr/bin/docker:/usr/bin/docker -v /var/run/docker.sock:/var/run/docker.sock --name jenkins20170204 jenkins20170204:latest登陆到jenkins容器中，查看jenkins 进程是否启动[root@docker ~]# docker exec -it jenkins20170204 /bin/bash

测试docker 是否可用

解决办法：yum install libtool-ltdl-devel再次执行，查看启动的容器

查看镜像

打开浏览器，通过jenkins，把build-nginx git到jenkins本地后，打包成镜像。

注意 $WORKSPACE是指定的 git 下载的路径名称：比如 build-nginx

开始构建

已经生成镜像。

java项目实现流程

1.jenkins 新建一个项目

2.从git仓库把项目克隆到本地

3.通过docker构建成镜像

4.上传到registry服务器

5.client 通过 docker pull registry镜像到本地，启动容器。

6.测试环境，研发环境，通过images传递。

分成三层： 1. 基础镜像

2. 中间件镜像

3. 应用镜像

构建java项目docker镜像，首先 把java 用maven 编译，并且构建成镜像，通过 docker cp命令把war包拷贝 到中间件。

FROM centos7.:
MAINTAINER fengjian <fengjian@senyint.com># Install maven
ADD apache-maven-3.3.-bin.tar.gz /data/
ADD jdk.tar.gz /data
#COPY tomcat /data/tomcat
#COPY jenkins.war /data/tomcat/webapps/COPY apache-maven-3.3. /data/maven
COPY settings.xml /data/maven/conf/settings.xml
CMD ["source /etc/profile"]
#ADD supervisor_tomcat.conf /etc/supervisor.conf.d/tomcat.confADD hello /hello
RUN cd /hello && \
/data/maven/bin/mvn install package

.构建maven镜像，已经通过mvn install package 编译java代码
[root@docker ~]# docker build -t senyint/maven:v1 .创建 maven 容器，但是不启动
[root@docker ~]# docker create --name maven senyint/maven:v1.把hello.war 的包 从maven容器中拷贝出来
[root@docker ~]# docker cp maven:/hello/target/hello.war .

#########################################################################################################################################################################################################

自己总结

分成三层： 1. 基础镜像                  centos:7.3      supervisor

2. 中间件镜像               java  maven    tomcat

3. 应用镜像                  java项目 war包 (jenkins   ........)

1. centos7.3 基础镜像

Dockerfile centos7.3基础镜像FROM centos:latest
MAINTAINER fengjian <fengjian@senyint.com>ENV TZ "Asia/Shanghai"
ENV TERM xtermADD .repo /etc/yum.repos.d/.repo
ADD aliyun-mirror.repo /etc/yum.repos.d/CentOS-Base.repo
ADD aliyun-epel.repo /etc/yum.repos.d/epel.repoRUN yum install -y curl openssl* wget libtool-ltdl-devel tar bzip2 unzippasswd sudo yum-utils hostname net-tools && \
yum install -y gcc gcc-c++ git make automake cmake patch logrotate python-devel libpng-devel libjpeg-devel && \
yum install -y --enablerepo=epel pwgen python-pip && \
yum clean allRUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple supervisor
ADD supervisord.conf /etc/supervisord.confRUN mkdir -p /etc/supervisor.conf.d && \
mkdir -p /var/log/supervisorENTRYPOINT ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]

supervisord.conf 配置文件[root@docker centos7]# cat supervisord.conf
[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod= ; socket file mode (default )[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB
logfile_backup=
loglevel=info
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=true ; (Start in foreground if true; default false)
minfds= ; (min. avail startup file descriptors;default )
minprocs= ; (min. avail process descriptors;default )[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket[include]
files = /etc/supervisor.conf.d/*.conf

二.中间件镜像

[root@docker jdk]# ls
Dockerfile jdk.tar.gz maven.tar.gz profile supervisor_tomcat.conf tomcat.tar.gz

[root@docker jdk]# vim profile 最底部添加环境变量export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$JAVA_HOME:$PATHMAVEN_HOME=/data/maven
export MAVEN_HOME
export PATH=${PATH}:${MAVEN_HOME}/bin

[root@docker jdk]# vim supervisor_tomcat.conf 用于启动tomcat[program:tomcat]
directory=/
command=/data/tomcat/bin/catalina.sh start
user=root
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.log

[root@docker jdk]# vim Dockerfile

FROM senyint/centos7.3
MAINTAINER fengjian <fengjian@senyint.com.com>

ENV JAVA_HOME /data/jdk
ENV JRE_HOME ${JAVA_HOME}/jre
ENV CLASSPATH .:${JAVA_HOME}/lib:${JRE_HOME}/lib

ENV MAVEN_HOME /data/maven
ENV PATH ${PATH}:${MAVEN_HOME}/bin:$JAVA_HOME/bin:$JRE_HOME/bin:$JAVA_HOME:$PATH

RUN mkdir -p /data/webserver
ADD maven.tar.gz /data
ADD jdk.tar.gz /data
ADD tomcat.tar.gz /data
ADD profile /etc
#ADD env.sh /etc/profile.d/

ADD supervisord.conf /etc/supervisord.conf
ADD supervisor_tomcat.conf /etc/supervisor.conf.d/tomcat.conf

RUN mkdir -p /etc/supervisor.conf.d && \
mkdir -p /var/log/supervisor

ENTRYPOINT ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]

 

 

FROM docker.cinyi.com:/centos7.#维护者
MAINTAINER fengjian <fengjian@senyint.com>
#设置一个时区的环境变量
ENV TZ "Asia/Shanghai"
#虚拟终端
ENV TERM xtermENV JAVA_HOME /data/jdk
ENV JRE_HOME ${JAVA_HOME}/jre
ENV CLASSPATH .:${JAVA_HOME}/lib:${JRE_HOME}/libENV MAVEN_HOME /data/maven
ENV PATH ${PATH}:${MAVEN_HOME}/bin:$JAVA_HOME/bin:$JRE_HOME/bin:$JAVA_HOME:$PATHRUN mkdir -p /data/webserver
ADD jdk.tar.gz /data
ADD tomcat.tar.gz /data
Add host.sh /dataADD profile /etc
RUN chmod +x /data/host.sh ; /data/host.shEXPOSE ENTRYPOINT ["/data/tomcat/bin/catalina.sh", "run" ]

 

[root@docker jdk]# docker build -t senyint/tomcat:v1 .
启动容器测试 java 环境变量
[root@docker jdk]# docker run -d -p : --name tomcat1 senyint/tomcat:v1
[root@docker jdk]# docker exec -it tomcat1 /bin/bash
[root@docker jdk]# java -version 显示java版本
[root@docker jdk]# mvn -version 显示maven版本

3构建应用（1）构建jenkins

jenkins Dockerfile文件

[root@docker jenkins]# vim Dockerfile

FROM senyint/java1.8:latest

MAINTAINER fengjian <fengjian@senyint.com.com>

ENV JENKINS_HOME /data/jenkins_home

ADD profile /etc/

ADD jenkins.war /data/webserver/

RUN unzip /data/webserver/jenkins.war -d /data/webserver && \
rm /data/webserver/jenkins.war

VOLUME /data/jenkins_home

构建jenkins镜像
[root@docker jenkins]# docker build -t senyint/jenkins .启动 jenkins 并且登陆到容器中
[root@docker jenkins]# docker run -d -p 11111:80 -v /usr/bin/docker:/usr/bin/docker -v /var/run/docker.sock:/var/run/docker.sock -v /docker_project:/docker_project --name jenkins senyint/jenkins

-v /docker_project:/docker_project 这个是把/docker_project 目录映射到 jenkins 容器中， jenkins容器 编译完war包后，通过cp命令拷贝到 /docker_project/java工程目录/ ,通过senyint/tomcat镜像构建java项目镜像。

[root@docker jenkins]# docker exec -it jenkins/bin/bash使用docker命令，

以下为jenkins 配置，以及 docker 对im-web项目进行编译，构建im-web镜像，并且推送到registry.

从git 仓库 下载java项目代码

构建时出现操作方法

registry="docker.cinyi.com:443"
#取出项目目录
javadir=`echo $WORKSPACE | awk -F'/' '{print $5}'`
#取出war包名称
javaname=`ls $WORKSPACE/target/*war | awk -F'/' '{print $7}' | cut -d . -f 1`

mkdir -p /data/docker_project/$javadir
rm /data/docker_profile/$javadir/$javaname.war -rf
mv $WORKSPACE/target/$javaname.war /data/docker_project/$javadir

#在/data/docker_project 目录下有一个dockerfile模版，根据war包的名字替换成新的dockerfile
sed "s/jenkins/$javaname/g" /data/docker_project/Dockerfile >/data/docker_project/$javadir/Dockerfile

if docker images | grep $javaname ; then
docker rmi -f docker.cinyi.com:443/senyint/$javaname
fi

docker build -t docker.cinyi.com:443/senyint/$javaname /data/docker_project/$javadir/
docker push docker.cinyi.com:443/senyint/$javaname

#定义namespace 为test：

k8s_apicurl="curl --cacert /root/ca.pem"
k8s_url="https://192.168.20.227:6443"

#创建namespaces
if ! `$k8s_apicurl -H "Authorization: Bearer 199e9c8d4ce99c61" -X GET $k8s_url/api/v1/namespaces | grep test >/dev/null` ;then
$k8s_apicurl -H "Authorization: Bearer 199e9c8d4ce99c61" -H "content-Type: application/yaml" -X POST $k8s_url/api/v1/namespaces -d "$(cat /root/namespaces.yaml)"
fi

#创建service
if ! `$k8s_apicurl -H "Authorization: Bearer 199e9c8d4ce99c61" -X GET $k8s_url/api/v1/namespaces/test/services | grep "im-web" >/dev/null` ; then
$k8s_apicurl -H "Authorization: Bearer 199e9c8d4ce99c61" -H "content-Type: application/yaml" -X POST $k8s_url/api/v1/namespaces/test/services -d "$(cat /root/im-web_service.yaml)"
fi

#创建deployment
if ! `$k8s_apicurl -H "Authorization: Bearer 199e9c8d4ce99c61" -X GET $k8s_url/apis/extensions/v1beta1/namespaces/test/deployments | grep "im-web" >/dev/null` ; then
$k8s_apicurl -H "Authorization: Bearer 199e9c8d4ce99c61" -H "content-Type: application/yaml" -X POST $k8s_url/apis/extensions/v1beta1/namespaces/test/deployments/ -d "$(cat /root/im-web_deployment.yaml)"
fi

把docker 命令 sock 直接挂载到容器中，进行打包，做成镜像。

docker run -d -p 80:80 --restart=always -v /usr/bin/docker:/usr/bin/docker -v /var/run/docker.sock:/var/run/docker.sock -v /data/docker_project:/data/docker_project  -v /data/jenkins_home:/data/jenkins_home -v /etc/sysconfig/docker:/etc/sysconfig/docker  senyint/jenkins

登陆到容器中，出现docker-client， docker版本是1.12.6

[root@6882772021f0 /]# docker ps -a
You don't have either docker-client or docker-client-latest installed. Please install either one and retry.

1.13版本好像没有问题，1.12.6在容器中yum install docker-client

[root@6882772021f0 /]# yum -y install docker-client

###################################################################################

docker资源隔离 使用linux LXC容器技术，主要是使用namespace(命名空间)kernel namespace（资源隔离） 分为
1.PID，通过pid 隔离，容器有自己独立的进程表和1号进程
2.net，通过网络隔离，容器有自己独立的network info
3.ipc，进程间的交互方法,在ipc通信时，需要加入额外信息来标识进程
4.mnt， 类似chroot，每个容器有自己唯一的目录挂载
5.uts， 可以让容器拥有自己的hostname 和domain
6.user， 可以拥有不通的用户，组

docker 网络模式. Nat 网络地址转换. Host

创建host网络， 和宿主机的网络相同
[root@docker data]# docker run -d --name centos7-host --net=host centos7.3:20170204.other container

容器之间相互通信十分频繁，可以使用这种模式，container网络特点

1.与主机网络空间隔离

2.容器间共享网络空间

3.适合容器间网络通信频繁。

[root@docker data]# docker run -d --name centos7-nat centos7.3:20170204 nat模式
[root@docker data]# docker run -d --name centos-container --net=container:centos7-nat  centos7.3:20170204centos-container容器的 ip地址 与 centos7-nat 的地址相同

. none
docker 容器 无网络配置，可自行配置。
[root@docker data]# docker run -d --name centos-none --net=none  centos7.3:20170204登陆到容器后，没有eth0

. overlay

overlay 网络特点

1. 跨主机通信

2.无需做端口管理

3.无需担心IP冲突

Consul介绍 

Consul 提供了分布式系统的服务发现和配置的解决方案。基于go语言实现。并且在git上开放了源码。consul还包括了分布式一致协议的实现，健康检查和管理UI.

Consul Agent Server、Client介绍

通过运行 consul agent 命令，可以通过后台守护进程的方式运行在所有consul集群节点中。以server或者client 模式运行。并且以HTTP或者DNS 接口方式，

负责运行检查和服务同步。Server模式的agent负责维护consul集群状态，相应RPC查询，并且还要负责和其他数据中心进行WAN Gossips交换。client 节点是

相对无状态的，Client的唯一活动就是转发请求给Server节点，以保持低延迟和少资源消耗。

openvswitch docker 配置

测试环境 ens32:192.168.20.209 docker0:10.0.1.1/24
ens32:192.168.20.135 docker0:10.0.2.1/24
   ens32:192.168.20.223 docker0:10.0.3.1/24
       ens32:192.168.20.224  docker0:10.0.4.1/24

192.168.20.20服务器操作
[root@docker ~]# rpm -ivh openvswitch-2.5.0-2.el7.x86_64.rpm
warning: openvswitch-2.5.0-2.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID fac8d3c0: NOKEY
error: Failed dependencies:
libatomic.so.1()(64bit) is needed by openvswitch-2.5.0-2.el7.x86_64

[root@docker ~]# yum -y install libatomic

[root@docker ~]# systemctl start openvswitch.service

[root@docker ~]# systemctl status openvswitch.service
openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; disabled; vendor preset: disabled)
Active: active (exited) since Sat 2017-02-11 10:00:11 CST; 17s ago
Process: 3854 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 3854 (code=exited, status=0/SUCCESS)

Feb 11 10:00:11 docker systemd[1]: Starting Open vSwitch...
Feb 11 10:00:11 docker systemd[1]: Started Open vSwitch.

[root@docker ~]# yum -y install bridge-utils

[root@docker ~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cno

#创建一个新的网桥
[root@docker ~]# ovs-vsctl add-br br0

#将网络设备端口添加到桥接器
[root@docker ~]# ovs-vsctl add-br br0
[root@docker ~]# ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre option:remote_ip=192.168.20.135

包括以下命令
[root@docker ~]# ovs-vsctl show
6fde4aed-708a-4ecc-882a-a415b3b3ac3d
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "gre0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.20.135"}
ovs_version: "2.5.0"

#[root@docker ~]# ovs-vsctl del-br br0

#添加br0到本地docker0,使容器流量通过openvswitc的隧道流出
[root@docker ~]# brctl addif docker0 br0
[root@docker ~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cnobr0

[root@docker ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:84:42:8d brd ff:ff:ff:ff:ff:ff
inet 192.168.20.209/24 brd 192.168.20.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe84:428d/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
link/ether 02:42:fa:be:52:1c brd ff:ff:ff:ff:ff:ff
inet 10.0.1.1/24 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:faff:febe:521c/64 scope link
valid_lft forever preferred_lft forever
422: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether c6:82:46:70:bc:d1 brd ff:ff:ff:ff:ff:ff
424: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN
link/gre 0.0.0.0 brd 0.0.0.0
425: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
426: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master docker0 state DOWN
link/ether 8e:63:55:ec:3b:41 brd ff:ff:ff:ff:ff:ff

#启用docker0 和 br0 网卡
[root@docker ~]# ip link set dev br0 up
[root@docker ~]# ip link set dev docker0 up

添加一条路由，所有去10.0.0.0/8 的网络从docker0出
[root@docker ~]# ip route add 10.0.0.0/8 dev docker0

启动一个容器
[root@docker ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
centos7.3 20170204 1d6f132807d0 6 days ago 530 MB

[root@docker ~]# docker run -d --name 209test centos7.3:20170204
登录到容器中
[root@docker ~]# docker exec -it 209test /bin/bash
查看ip地址
[root@464241f535e2 /]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.1.2 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::42:aff:fe00:102 prefixlen 64 scopeid 0x20<link>
ether 02:42:0a:00:01:02 txqueuelen 0 (Ethernet)
RX packets 720 bytes 68496 (66.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 693 bytes 65706 (64.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

#ping 192.168.20.135 服务器容器的IP地址
[root@464241f535e2 /]# ping 10.0.2.2

PING 10.0.2.2 (10.0.2.2) 56(84) bytes of data.
64 bytes from 10.0.2.2: icmp_seq=1 ttl=63 time=0.451 ms
64 bytes from 10.0.2.2: icmp_seq=2 ttl=63 time=0.493 ms
From 10.0.1.1 icmp_seq=3 Redirect Host(New nexthop: 10.0.2.2)
From 10.0.1.1: icmp_seq=3 Redirect Host(New nexthop: 10.0.2.2)

出现以上问题，过一会，就恢复正常。

##############################################################################################

192.168.20.135、223、224 服务器操作
[root@registry ~]# rpm -ivh openvswitch-2.5.0-2.el7.x86_64.rpm
warning: openvswitch-2.5.0-2.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID fac8d3c0: NOKEY
error: Failed dependencies:
libatomic.so.1()(64bit) is needed by openvswitch-2.5.0-2.el7.x86_64

[root@registry ~]# yum -y install libatomic

[root@registry ~]# systemctl start openvswitch.service

[root@registry ~]# systemctl status openvswitch.service
● openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; disabled; vendor preset: disabled)
Active: active (exited) since Sat 2017-02-11 10:00:11 CST; 17s ago
Process: 3854 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 3854 (code=exited, status=0/SUCCESS)

Feb 11 10:00:11 registry systemd[1]: Starting Open vSwitch...
Feb 11 10:00:11 registry systemd[1]: Started Open vSwitch.

[root@registry ~]# yum -y install bridge-utils

[root@registry ~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cno

#创建一个新的网桥
[root@registry ~]# ovs-vsctl add-br br0

#将网络设备端口添加到桥接器
[root@registry ~]# ovs-vsctl add-br br0
[root@registry ~]# ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre option:remote_ip=192.168.20.209

包括以下命令，ovs-vsctl show 的结果与 192.168.20.209不一样
[root@registry ~]# ovs-vsctl show
19baf011-40aa-426c-a2b9-568101390834
Bridge "br0"
Port "gre0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.20.209"}
Port "br0"
Interface "br0"
type: internal
ovs_version: "2.5.0"

####删除br0 [root@registry ~]# ovs-vsctl del-br br0

#添加br0到本地registry0,使容器流量通过openvswitc的隧道流出
[root@registry ~]# brctl addif docker0 br0
[root@registry ~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cnobr0

[root@registry ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:84:2b:fc brd ff:ff:ff:ff:ff:ff
inet 192.168.20.135/24 brd 192.168.20.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe84:2bfc/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state DOWN
link/ether 02:42:5e:5d:06:3f brd ff:ff:ff:ff:ff:ff
inet 10.0.2.1/24 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:5eff:fe5d:63f/64 scope link
valid_lft forever preferred_lft forever
28: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether f2:fd:f4:39:e2:20 brd ff:ff:ff:ff:ff:ff
29: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state DOWN
link/ether 0a:29:1e:93:37:41 brd ff:ff:ff:ff:ff:ff
inet6 fe80::829:1eff:fe93:3741/64 scope link
valid_lft forever preferred_lft forever

#启用docker0 和 br0 网卡
[root@registry ~]# ip link set dev br0 up
[root@registry ~]# ip link set dev docker0 up

添加一条路由，所有去10.0.0.0/8 的网络从docker0出
[root@registry ~]# ip route add 10.0.0.0/8 dev docker0

#########################################################################################

192.168.20.223 服务器操作
[root@docker223~]# rpm -ivh openvswitch-2.5.0-2.el7.x86_64.rpm 
warning: openvswitch-2.5.0-2.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID fac8d3c0: NOKEY
error: Failed dependencies:
libatomic.so.1()(64bit) is needed by openvswitch-2.5.0-2.el7.x86_64

[root@docker223~]# yum -y install libatomic

[root@docker223~]# systemctl start openvswitch.service

[root@docker223~]# systemctl status openvswitch.service 
● openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; disabled; vendor preset: disabled)
Active: active (exited) since Sat 2017-02-11 10:00:11 CST; 17s ago
Process: 3854 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 3854 (code=exited, status=0/SUCCESS)

Feb 11 10:00:11 docker223systemd[1]: Starting Open vSwitch...
Feb 11 10:00:11 docker223systemd[1]: Started Open vSwitch.

[root@docker223~]# yum -y install bridge-utils

[root@docker223~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cno

#创建一个新的网桥
[root@docker223~]# ovs-vsctl add-br br0

#将网络设备端口添加到桥接器
[root@docker223~]# ovs-vsctl add-br br0
[root@docker223~]# ovs-vsctl add-port br0 gre1 -- set interface gre1 type=gre option:remote_ip=192.168.20.209

包括以下命令，ovs-vsctl show 的结果与 192.168.20.209不一样
[root@docker223~]# ovs-vsctl show

8256b14a-1da6-4781-b9aa-7c6612ce7ebf
Bridge "br0"
Port "gre1"
Interface "gre1"
type: gre
options: {remote_ip="192.168.20.209"}
Port "br0"
Interface "br0"
type: internal
ovs_version: "2.5.0"

####删除br0 [root@docker223~]# ovs-vsctl del-br br0

#添加br0到本地registry0,使容器流量通过openvswitc的隧道流出
[root@docker223~]# brctl addif docker0 br0
[root@docker223~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cnobr0

[root@docker223~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:84:2b:fc brd ff:ff:ff:ff:ff:ff
inet 192.168.20.223/24 brd 192.168.20.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe84:2bfc/64 scope link 
valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state DOWN 
link/ether 02:42:5e:5d:06:3f brd ff:ff:ff:ff:ff:ff
inet 10.0.3.1/24 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:5eff:fe5d:63f/64 scope link 
valid_lft forever preferred_lft forever
28: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
link/ether f2:fd:f4:39:e2:20 brd ff:ff:ff:ff:ff:ff
29: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state DOWN 
link/ether 0a:29:1e:93:37:41 brd ff:ff:ff:ff:ff:ff
inet6 fe80::829:1eff:fe93:3741/64 scope link 
valid_lft forever preferred_lft forever

#启用docker0 和 br0 网卡
[root@docker223~]# ip link set dev br0 up
[root@docker223~]# ip link set dev docker0 up

添加一条路由，所有去10.0.0.0/8 的网络从docker0出
[root@docker223~]# ip route add 10.0.0.0/8 dev docker0

#########################################################################################

192.168.20.224 服务器操作
[root@docker224~]# rpm -ivh openvswitch-2.5.0-2.el7.x86_64.rpm 
warning: openvswitch-2.5.0-2.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID fac8d3c0: NOKEY
error: Failed dependencies:
libatomic.so.1()(64bit) is needed by openvswitch-2.5.0-2.el7.x86_64

[root@docker224~]# yum -y install libatomic

[root@docker224~]# systemctl start openvswitch.service

[root@docker224~]# systemctl status openvswitch.service 
● openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; disabled; vendor preset: disabled)
Active: active (exited) since Sat 2017-02-11 10:00:11 CST; 17s ago
Process: 3854 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 3854 (code=exited, status=0/SUCCESS)

Feb 11 10:00:11 docker223systemd[1]: Starting Open vSwitch...
Feb 11 10:00:11 docker223systemd[1]: Started Open vSwitch.

[root@docker223~]# yum -y install bridge-utils

[root@docker224~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cno

#创建一个新的网桥
[root@docker224~]# ovs-vsctl add-br br0

#将网络设备端口添加到桥接器
[root@docker224~]# ovs-vsctl add-br br0
[root@docker224~]# ovs-vsctl add-port br0 gre1 -- set interface gre1 type=gre option:remote_ip=192.168.20.209

包括以下命令，ovs-vsctl show 的结果与 192.168.20.209不一样
[root@docker224~]# ovs-vsctl show

8256b14a-1da6-4781-b9aa-7c6612ce7ebf
Bridge "br0"
Port "gre2"
Interface "gre2"
type: gre
options: {remote_ip="192.168.20.209"}
Port "br0"
Interface "br0"
type: internal
ovs_version: "2.5.0"

####删除br0 [root@docker224~]# ovs-vsctl del-br br0

#添加br0到本地docker0,使容器流量通过openvswitc的隧道流出
[root@docker224~]# brctl addif docker0 br0
[root@docker224~]# brctl show
bridge namebridge idSTP enabledinterfaces
docker08000.0242fabe521cnobr0

[root@docker224~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:84:2b:fc brd ff:ff:ff:ff:ff:ff
inet 192.168.20.224/24 brd 192.168.20.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe84:2bfc/64 scope link 
valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state DOWN 
link/ether 02:42:5e:5d:06:3f brd ff:ff:ff:ff:ff:ff
inet 10.0.4.1/24 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:5eff:fe5d:63f/64 scope link 
valid_lft forever preferred_lft forever
28: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
link/ether f2:fd:f4:39:e2:20 brd ff:ff:ff:ff:ff:ff
29: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state DOWN 
link/ether 0a:29:1e:93:37:41 brd ff:ff:ff:ff:ff:ff
inet6 fe80::829:1eff:fe93:3741/64 scope link 
valid_lft forever preferred_lft forever

#启用docker0 和 br0 网卡
[root@docker224~]# ip link set dev br0 up
[root@docker224~]# ip link set dev docker0 up

添加一条路由，所有去10.0.0.0/8 的网络从docker0出
[root@docker224~]# ip route add 10.0.0.0/8 dev docker0

总结，192.168.20.209 在br0 上添加 gre0 gre1 gre2 ,对应关系如下， 而192.168.20.135 br0添加gre0， 192.168.20.223 添加gre1， 192.168.20.224 添加gre2， 之后分别启动容器，不同容器的网段 可以相互ping通。

gr0 192.168.20.135
gr1 192.168.20.223
gr2 192.168.20.224

192.168.20.209自启动脚本

#!/bin/bash

systemctl start openvswitch.service

systemctl enable openvswitch.service

ovs-vsctl add-br br0
ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre option:remote_ip=192.168.20.135
ovs-vsctl add-port br0 gre1 -- set interface gre1 type=gre option:remote_ip=192.168.20.223
ovs-vsctl add-port br0 gre2 -- set interface gre2 type=gre option:remote_ip=192.168.20.224

ovs-vsctl show

brctl show

brctl addif docker0 br0

ip link set dev br0 up

ip link set dev docker0 up

ip route add 10.0.0./8 dev docker0

 192.168.20.135自启动脚本

systemctl start openvswitch.service

systemctl enable openvswitch.service

ovs-vsctl add-br br0
ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre option:remote_ip=192.168.20.209

ovs-vsctl show

brctl show

brctl addif docker0 br0

ip link set dev br0 up

ip link set dev docker0 up

ip route add 10.0.0.0/8 dev docker0

 192.168.20.223自启动脚本

systemctl start openvswitch.service

systemctl enable openvswitch.service

ovs-vsctl add-br br0
ovs-vsctl add-port br0 gre1 -- set interface gre1 type=gre option:remote_ip=192.168.20.209

ovs-vsctl show

brctl show

brctl addif docker0 br0

ip link set dev br0 up

ip link set dev docker0 up

ip route add 10.0.0.0/8 dev docker0

 192.168.20.224自启动脚本

systemctl start openvswitch.service

systemctl enable openvswitch.service

ovs-vsctl add-br br0
ovs-vsctl add-port br0 gre1 -- set interface gre1 type=gre option:remote_ip=192.168.20.209

ovs-vsctl show

brctl show

brctl addif docker0 br0

ip link set dev br0 up

ip link set dev docker0 ip

ip route add 10.0.0.0/8 dev docker0