正文

[补充资料] 手动搭建 Cloudera 集群

IP云
IP云V管理员/10 阅读
09/01
提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

本课主题

  • 集群搭建
  • 设置 Web 服务器
  • 启动 ClouderManager
  • 登入 Cloudera Manager

引言

这部份是一个补充资料，记录如何安装 Cloudera 服务器

集群搭建

  1. 查看你主机的 IP 地址并且设置静态 IP，这里显示的是 192.168.80.145

    ifconfig -a


    用图型界面来设置静态 IP

  2. 修改 network 文件来修改 HOSTNAME 

    vim /etc/sysconfig/network
    hostname elephant
    service network restart

  3. 修改 /etc/hosts 文件

    192.168.80.145 elephant
    192.168.80.146 monkey 

设置 Web 服务器

  1. 首先是关掉防火墙

    service iptables stop
    chkconfig iptables off
    vim /etc/selinux/config
    setenforce 0

  2. 查看 httpd 服务和启动状况，如果发现没有安装 httpd 服务，请先安装 sudo yum -y install httpd

    [enmoedu@elephant softwares]$ rpm -qa | grep httpd
    httpd-2.2.15-59.el6.centos.x86_64
    httpd-tools-2.2.15-59.el6.centos.x86_64

    rpm -qa | grep httpd

    [root@elephant softwares]# service httpd status
    httpd is stopped
    [root@elephant softwares]# service httpd start
    Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 192.168.80.145 for ServerName
    [ OK ]
    [root@elephant softwares]# service httpd status
    httpd (pid 3002) is running...

    service httpd status

  3. 查看 WEB 服务器的状况: http://elephant
  4. 安装好之后把 cloudera-cdh5 和 cloudera-manager 移动到 /var/www/html/ 文件夹里。 

    http://elephant/cloudera-manager5.9
    http://elephant/cloudera-cdh5.9 

    此时你可以创建安装包啦，进入 cloudera-cdh5 和 cloudera-manager 创建 repo

    createrepo .

    [root@elephant html]# cd cloudera-cdh5.9/
    [root@elephant cloudera-cdh5.9]# ls
    repodata RPMS
    [root@elephant cloudera-cdh5.9]# createrepo .
    Spawning worker 0 with 117 pkgs
    Workers Finished
    Gathering worker resultsSaving Primary metadata
    Saving file lists metadata
    Saving other metadata
    Generating sqlite DBs
    Sqlite DBs complete
    [root@elephant cloudera-cdh5.9]# cd ..
    [root@elephant html]# ls
    cloudera-cdh5.9 cloudera-manager5.9
    [root@elephant html]# cd cloudera-manager5.9/
    [root@elephant cloudera-manager5.9]# ls
    repodata RPMS
    [root@elephant cloudera-manager5.9]# createrepo .
    Spawning worker 0 with 6 pkgs
    Workers Finished
    Gathering worker resultsSaving Primary metadata
    Saving file lists metadata
    Saving other metadata
    Generating sqlite DBs
    Sqlite DBs complete

    createrepo

  5. 完成以上的步骤，服务器已经搭建好了，剩下来是客户端的配置。创建好 repo 之后可以到 /etc/yum.repos.d/ 文件夹创建 repo文件，新建 cloudera-cdh5 的 yum 源。

    [Cloudera-cdh5]
    name=Cloudera-cdh5
    baseurl=http://elephant/Cloudera-cdh5
    gpgcheck=0
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

    Cloudera-cdh5.repo

    [cloudera-manager]
    name=cloudera-manager
    baseurl=http://elephant/cloudera-manager
    gpgcheck=0
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

    cloudera-manager.repo

    mv Cloudera-cdh5/ cloudera-manager/ /var/www/html/
    service httpd status
    cd /var/www/html/cd Cloudera-cdh5/
    createrepo .cd cloudera-manager/
    createrepo .cd /etc/yum.repos.d/ cat << EOF > /etc/yum.repos.d/Cloudera-cdh5.repo
    [Cloudera-cdh5]
    name=Cloudera-cdh5
    baseurl=http://elephant/Cloudera-cdh5
    gpgcheck=0
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
    EOFcat << EOF > /etc/yum.repos.d/cloudera-manager.repo
    [cloudera-manager]
    name=cloudera-manager
    baseurl=http://elephant/cloudera-manager
    gpgcheck=0
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
    EOF

    completed-code

    清理一下 yum

    [root@elephant yum.repos.d]# yum clean all
    Loaded plugins: fastestmirror, refresh-packagekit
    Cleaning repos: base cloudera-cdh5.9 cloudera-manager extras updates
    Cleaning up Everything
    Cleaning up list of fastest mirrors

    yum clean all

    [root@elephant yum.repos.d]# yum install spark-core
    Loaded plugins: fastestmirror, refresh-packagekit
    Setting up Install Process
    Determining fastest mirrors
    * base: centos.01link.hk
    * extras: centos.01link.hk
    * updates: centos.01link.hk
    base | 3.7 kB 00:00
    base/primary_db | 4.7 MB 00:02
    cloudera-cdh5.9 | 2.9 kB 00:00
    cloudera-cdh5.9/primary_db | 64 kB 00:00
    cloudera-manager | 2.9 kB 00:00
    cloudera-manager/primary_db | 8.8 kB 00:00
    extras | 3.4 kB 00:00
    extras/primary_db | 37 kB 00:00
    updates | 3.4 kB 00:00
    updates/primary_db | 803 kB 00:00
    Resolving Dependencies
    --> Running transaction check
    ---> Package spark-core.noarch 0:1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6 will be installed
    base/filelists_db | 6.4 MB 00:02
    cloudera-cdh5.9/filelists_db | 413 kB 00:00
    cloudera-manager/filelists_db | 94 kB 00:00
    extras/filelists_db | 38 kB 00:00
    updates/filelists_db | 644 kB 00:00
    --> Processing Dependency: bigtop-utils >= 0.7 for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: parquet for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: hive for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: hadoop-client for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: flume-ng for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: avro-libs for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: /lib/lsb/init-functions for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch
    --> Running transaction check
    ---> Package avro-libs.noarch 0:1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package bigtop-utils.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package flume-ng.noarch 0:1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: kite >= 0.10.0+cdh5.1.0 for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: hive-jdbc >= 1.1.0+cdh5.4.0 for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: hadoop-yarn for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: hadoop-mapreduce for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: hadoop-hdfs for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch
    --> Processing Dependency: hadoop for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch
    ---> Package hadoop-client.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: hadoop-0.20-mapreduce = 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 for package: hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64
    --> Processing Dependency: hadoop-0.20-mapreduce = 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 for package: hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64
    --> Processing Dependency: zookeeper for package: hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64
    ---> Package hive.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: sentry for package: hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch
    ---> Package parquet.noarch 0:1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: parquet-format >= 2.1.0 for package: parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch
    ---> Package redhat-lsb-core.x86_64 0:4.0-7.el6.centos will be installed
    --> Processing Dependency: perl-Test-Simple for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: perl-Test-Harness for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: perl-ExtUtils-MakeMaker for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: perl-CGI for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /usr/bin/time for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /usr/bin/pax for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /usr/bin/man for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /usr/bin/bc for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /usr/bin/batch for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /usr/bin/at for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /bin/mailx for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Processing Dependency: /bin/ed for package: redhat-lsb-core-4.0-7.el6.centos.x86_64
    --> Running transaction check
    ---> Package at.x86_64 0:3.1.10-49.el6 will be installed
    ---> Package bc.x86_64 0:1.06.95-1.el6 will be installed
    ---> Package ed.x86_64 0:1.1-3.3.el6 will be installed
    ---> Package hadoop.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: nc for package: hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64
    ---> Package hadoop-0.20-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package hadoop-hdfs.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: bigtop-jsvc for package: hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64
    ---> Package hadoop-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package hadoop-yarn.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package hive-jdbc.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package kite.noarch 0:1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: solr >= 4.4.0+cdh5.1.0 for package: kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch
    ---> Package mailx.x86_64 0:12.4-8.el6_6 will be installed
    ---> Package man.x86_64 0:1.6f-39.el6 will be installed
    ---> Package parquet-format.noarch 0:2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package pax.x86_64 0:3.4-10.1.el6 will be installed
    ---> Package perl-CGI.x86_64 0:3.51-144.el6 will be installed
    --> Processing Dependency: perl = 4:5.10.1-144.el6 for package: perl-CGI-3.51-144.el6.x86_64
    ---> Package perl-ExtUtils-MakeMaker.x86_64 0:6.55-144.el6 will be installed
    --> Processing Dependency: perl-devel for package: perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64
    ---> Package perl-Test-Harness.x86_64 0:3.17-144.el6 will be installed
    ---> Package perl-Test-Simple.x86_64 0:0.92-144.el6 will be installed
    ---> Package sentry.noarch 0:1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package time.x86_64 0:1.7-38.el6 will be installed
    ---> Package zookeeper.x86_64 0:3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6 will be installed
    --> Running transaction check
    ---> Package bigtop-jsvc.x86_64 0:0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package nc.x86_64 0:1.84-24.el6 will be installed
    ---> Package perl.x86_64 4:5.10.1-141.el6_7.1 will be updated
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: perl-IO-Compress-Base-2.021-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Compress-Raw-Zlib-2.021-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: perl-IO-Compress-Zlib-2.021-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: perl-Compress-Zlib-2.021-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 3:perl-version-0.77-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 4:perl-libs-5.10.1-141.el6_7.1.x86_64
    --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64
    ---> Package perl.x86_64 4:5.10.1-144.el6 will be an update
    ---> Package perl-devel.x86_64 4:5.10.1-144.el6 will be installed
    --> Processing Dependency: perl(ExtUtils::ParseXS) for package: 4:perl-devel-5.10.1-144.el6.x86_64
    --> Processing Dependency: gdbm-devel for package: 4:perl-devel-5.10.1-144.el6.x86_64
    --> Processing Dependency: db4-devel for package: 4:perl-devel-5.10.1-144.el6.x86_64
    ---> Package solr.noarch 0:4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6 will be installed
    --> Processing Dependency: bigtop-tomcat for package: solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch
    --> Running transaction check
    ---> Package bigtop-tomcat.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 will be installed
    ---> Package db4-devel.x86_64 0:4.7.25-22.el6 will be installed
    --> Processing Dependency: db4-cxx = 4.7.25-22.el6 for package: db4-devel-4.7.25-22.el6.x86_64
    --> Processing Dependency: db4 = 4.7.25-22.el6 for package: db4-devel-4.7.25-22.el6.x86_64
    --> Processing Dependency: libdb_cxx-4.7.so()(64bit) for package: db4-devel-4.7.25-22.el6.x86_64
    ---> Package gdbm-devel.x86_64 0:1.8.0-39.el6 will be installed
    ---> Package perl-Compress-Raw-Zlib.x86_64 1:2.021-141.el6_7.1 will be updated
    ---> Package perl-Compress-Raw-Zlib.x86_64 1:2.021-144.el6 will be an update
    ---> Package perl-Compress-Zlib.x86_64 0:2.021-141.el6_7.1 will be updated
    ---> Package perl-Compress-Zlib.x86_64 0:2.021-144.el6 will be an update
    ---> Package perl-ExtUtils-ParseXS.x86_64 1:2.2003.0-144.el6 will be installed
    ---> Package perl-IO-Compress-Base.x86_64 0:2.021-141.el6_7.1 will be updated
    ---> Package perl-IO-Compress-Base.x86_64 0:2.021-144.el6 will be an update
    ---> Package perl-IO-Compress-Zlib.x86_64 0:2.021-141.el6_7.1 will be updated
    ---> Package perl-IO-Compress-Zlib.x86_64 0:2.021-144.el6 will be an update
    ---> Package perl-Module-Pluggable.x86_64 1:3.90-141.el6_7.1 will be updated
    ---> Package perl-Module-Pluggable.x86_64 1:3.90-144.el6 will be an update
    ---> Package perl-Pod-Escapes.x86_64 1:1.04-141.el6_7.1 will be updated
    ---> Package perl-Pod-Escapes.x86_64 1:1.04-144.el6 will be an update
    ---> Package perl-Pod-Simple.x86_64 1:3.13-141.el6_7.1 will be updated
    ---> Package perl-Pod-Simple.x86_64 1:3.13-144.el6 will be an update
    ---> Package perl-libs.x86_64 4:5.10.1-141.el6_7.1 will be updated
    ---> Package perl-libs.x86_64 4:5.10.1-144.el6 will be an update
    ---> Package perl-version.x86_64 3:0.77-141.el6_7.1 will be updated
    ---> Package perl-version.x86_64 3:0.77-144.el6 will be an update
    --> Running transaction check
    ---> Package db4.x86_64 0:4.7.25-20.el6_7 will be updated
    --> Processing Dependency: db4 = 4.7.25-20.el6_7 for package: db4-utils-4.7.25-20.el6_7.x86_64
    ---> Package db4.x86_64 0:4.7.25-22.el6 will be an update
    ---> Package db4-cxx.x86_64 0:4.7.25-22.el6 will be installed
    --> Running transaction check
    ---> Package db4-utils.x86_64 0:4.7.25-20.el6_7 will be updated
    ---> Package db4-utils.x86_64 0:4.7.25-22.el6 will be an update
    --> Finished Dependency ResolutionDependencies Resolved==============================================================================================================================================================
    Package Arch Version Repository Size
    ==============================================================================================================================================================
    Installing:
    spark-core noarch 1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 112 M
    Installing for dependencies:
    at x86_64 3.1.10-49.el6 base 61 k
    avro-libs noarch 1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 45 M
    bc x86_64 1.06.95-1.el6 base 110 k
    bigtop-jsvc x86_64 0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 27 k
    bigtop-tomcat noarch 0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 7.5 M
    bigtop-utils noarch 0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 11 k
    db4-cxx x86_64 4.7.25-22.el6 base 589 k
    db4-devel x86_64 4.7.25-22.el6 base 6.6 M
    ed x86_64 1.1-3.3.el6 base 72 k
    flume-ng noarch 1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 52 M
    gdbm-devel x86_64 1.8.0-39.el6 base 26 k
    hadoop x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 27 M
    hadoop-0.20-mapreduce x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 36 M
    hadoop-client x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 34 k
    hadoop-hdfs x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 24 M
    hadoop-mapreduce x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 32 M
    hadoop-yarn x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 15 M
    hive noarch 1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 42 M
    hive-jdbc noarch 1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 46 M
    kite noarch 1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 114 M
    mailx x86_64 12.4-8.el6_6 base 235 k
    man x86_64 1.6f-39.el6 base 265 k
    nc x86_64 1.84-24.el6 base 57 k
    parquet noarch 1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 26 M
    parquet-format noarch 2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 467 k
    pax x86_64 3.4-10.1.el6 base 69 k
    perl-CGI x86_64 3.51-144.el6 base 210 k
    perl-ExtUtils-MakeMaker x86_64 6.55-144.el6 base 294 k
    perl-ExtUtils-ParseXS x86_64 1:2.2003.0-144.el6 base 46 k
    perl-Test-Harness x86_64 3.17-144.el6 base 232 k
    perl-Test-Simple x86_64 0.92-144.el6 base 113 k
    perl-devel x86_64 4:5.10.1-144.el6 base 424 k
    redhat-lsb-core x86_64 4.0-7.el6.centos base 25 k
    sentry noarch 1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 56 M
    solr noarch 4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 76 M
    time x86_64 1.7-38.el6 base 26 k
    zookeeper x86_64 3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 4.4 M
    Updating for dependencies:
    db4 x86_64 4.7.25-22.el6 base 564 k
    db4-utils x86_64 4.7.25-22.el6 base 131 k
    perl x86_64 4:5.10.1-144.el6 base 10 M
    perl-Compress-Raw-Zlib x86_64 1:2.021-144.el6 base 70 k
    perl-Compress-Zlib x86_64 2.021-144.el6 base 46 k
    perl-IO-Compress-Base x86_64 2.021-144.el6 base 70 k
    perl-IO-Compress-Zlib x86_64 2.021-144.el6 base 136 k
    perl-Module-Pluggable x86_64 1:3.90-144.el6 base 41 k
    perl-Pod-Escapes x86_64 1:1.04-144.el6 base 33 k
    perl-Pod-Simple x86_64 1:3.13-144.el6 base 213 k
    perl-libs x86_64 4:5.10.1-144.el6 base 579 k
    perl-version x86_64 3:0.77-144.el6 base 52 kTransaction Summary
    ==============================================================================================================================================================
    Install 38 Package(s)
    Upgrade 12 Package(s)Total download size: 736 M
    Is this ok [y/N]:y
    Downloading Packages:
    (1/50): at-3.1.10-49.el6.x86_64.rpm | 61 kB 00:00
    (2/50): avro-libs-1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6.noarch.rpm | 45 MB 00:00
    (3/50): bc-1.06.95-1.el6.x86_64.rpm | 110 kB 00:00
    (4/50): bigtop-jsvc-0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 27 kB 00:00
    (5/50): bigtop-tomcat-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch.rpm | 7.5 MB 00:00
    (6/50): bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch.rpm | 11 kB 00:00
    (7/50): db4-4.7.25-22.el6.x86_64.rpm | 564 kB 00:00
    (8/50): db4-cxx-4.7.25-22.el6.x86_64.rpm | 589 kB 00:00
    (9/50): db4-devel-4.7.25-22.el6.x86_64.rpm | 6.6 MB 00:02
    (10/50): db4-utils-4.7.25-22.el6.x86_64.rpm | 131 kB 00:00
    (11/50): ed-1.1-3.3.el6.x86_64.rpm | 72 kB 00:00
    (12/50): flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch.rpm | 52 MB 00:00
    (13/50): gdbm-devel-1.8.0-39.el6.x86_64.rpm | 26 kB 00:00
    (14/50): hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 27 MB 00:00
    (15/50): hadoop-0.20-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 36 MB 00:00
    (16/50): hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 34 kB 00:00
    (17/50): hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 24 MB 00:00
    (18/50): hadoop-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 32 MB 00:00
    (19/50): hadoop-yarn-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 15 MB 00:00
    (20/50): hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch.rpm | 42 MB 00:00
    (21/50): hive-jdbc-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch.rpm | 46 MB 00:00
    (22/50): kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch.rpm | 114 MB 00:01
    (23/50): mailx-12.4-8.el6_6.x86_64.rpm | 235 kB 00:00
    (24/50): man-1.6f-39.el6.x86_64.rpm | 265 kB 00:00
    (25/50): nc-1.84-24.el6.x86_64.rpm | 57 kB 00:00
    (26/50): parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch.rpm | 26 MB 00:00
    (27/50): parquet-format-2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6.noarch.rpm | 467 kB 00:00
    (28/50): pax-3.4-10.1.el6.x86_64.rpm | 69 kB 00:00
    (29/50): perl-5.10.1-144.el6.x86_64.rpm | 10 MB 00:05
    (30/50): perl-CGI-3.51-144.el6.x86_64.rpm | 210 kB 00:00
    (31/50): perl-Compress-Raw-Zlib-2.021-144.el6.x86_64.rpm | 70 kB 00:00
    (32/50): perl-Compress-Zlib-2.021-144.el6.x86_64.rpm | 46 kB 00:00
    (33/50): perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64.rpm | 294 kB 00:00
    (34/50): perl-ExtUtils-ParseXS-2.2003.0-144.el6.x86_64.rpm | 46 kB 00:00
    (35/50): perl-IO-Compress-Base-2.021-144.el6.x86_64.rpm | 70 kB 00:00
    (36/50): perl-IO-Compress-Zlib-2.021-144.el6.x86_64.rpm | 136 kB 00:00
    (37/50): perl-Module-Pluggable-3.90-144.el6.x86_64.rpm | 41 kB 00:00
    (38/50): perl-Pod-Escapes-1.04-144.el6.x86_64.rpm | 33 kB 00:00
    (39/50): perl-Pod-Simple-3.13-144.el6.x86_64.rpm | 213 kB 00:00
    (40/50): perl-Test-Harness-3.17-144.el6.x86_64.rpm | 232 kB 00:00
    (41/50): perl-Test-Simple-0.92-144.el6.x86_64.rpm | 113 kB 00:00
    (42/50): perl-devel-5.10.1-144.el6.x86_64.rpm | 424 kB 00:00
    (43/50): perl-libs-5.10.1-144.el6.x86_64.rpm | 579 kB 00:00
    (44/50): perl-version-0.77-144.el6.x86_64.rpm | 52 kB 00:00
    (45/50): redhat-lsb-core-4.0-7.el6.centos.x86_64.rpm | 25 kB 00:00
    (46/50): sentry-1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6.noarch.rpm | 56 MB 00:00
    (47/50): solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch.rpm | 76 MB 00:00
    (48/50): spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch.rpm | 112 MB 00:01
    (49/50): time-1.7-38.el6.x86_64.rpm | 26 kB 00:00
    (50/50): zookeeper-3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 4.4 MB 00:00
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total 35 MB/s | 736 MB 00:21
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
    Installing : avro-libs-1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6.noarch 1/62
    Installing : bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 2/62
    Installing : zookeeper-3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6.x86_64 3/62
    Updating : db4-4.7.25-22.el6.x86_64 4/62
    Updating : 1:perl-Pod-Escapes-1.04-144.el6.x86_64 5/62
    Updating : 4:perl-libs-5.10.1-144.el6.x86_64 6/62
    Updating : 1:perl-Pod-Simple-3.13-144.el6.x86_64 7/62
    Updating : 1:perl-Module-Pluggable-3.90-144.el6.x86_64 8/62
    Updating : 3:perl-version-0.77-144.el6.x86_64 9/62
    Updating : 4:perl-5.10.1-144.el6.x86_64 10/62
    Updating : 1:perl-Compress-Raw-Zlib-2.021-144.el6.x86_64 11/62
    Updating : perl-IO-Compress-Base-2.021-144.el6.x86_64 12/62
    Updating : perl-IO-Compress-Zlib-2.021-144.el6.x86_64 13/62
    Installing : perl-CGI-3.51-144.el6.x86_64 14/62
    Installing : bigtop-jsvc-0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6.x86_64 15/62
    Installing : bigtop-tomcat-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 16/62
    Installing : nc-1.84-24.el6.x86_64 17/62
    Installing : ed-1.1-3.3.el6.x86_64 18/62
    Installing : mailx-12.4-8.el6_6.x86_64 19/62
    Installing : man-1.6f-39.el6.x86_64 20/62
    Installing : db4-cxx-4.7.25-22.el6.x86_64 21/62
    Installing : db4-devel-4.7.25-22.el6.x86_64 22/62
    Installing : gdbm-devel-1.8.0-39.el6.x86_64 23/62
    Installing : perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64 24/62
    Installing : perl-Test-Harness-3.17-144.el6.x86_64 25/62
    Installing : 1:perl-ExtUtils-ParseXS-2.2003.0-144.el6.x86_64 26/62
    Installing : 4:perl-devel-5.10.1-144.el6.x86_64 27/62
    Installing : perl-Test-Simple-0.92-144.el6.x86_64 28/62
    Installing : time-1.7-38.el6.x86_64 29/62
    Installing : bc-1.06.95-1.el6.x86_64 30/62
    Installing : pax-3.4-10.1.el6.x86_64 31/62
    Installing : at-3.1.10-49.el6.x86_64 32/62
    Installing : redhat-lsb-core-4.0-7.el6.centos.x86_64 33/62
    Installing : parquet-format-2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6.noarch 34/62
    Installing : parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch 35/62
    Installing : hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 36/62
    Installing : hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 37/62
    Installing : hadoop-yarn-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 38/62
    Installing : hadoop-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 39/62
    Installing : hadoop-0.20-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 40/62
    Installing : hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 41/62
    Installing : hive-jdbc-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 42/62
    Installing : hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 43/62
    Installing : sentry-1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6.noarch 44/62
    Installing : solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch 45/62
    The following warning applies to any collections configured to
    use Non-SolrCloud mode. Any such collection configuration will
    need to be upgraded, see Upgrading Cloudera Search for details.
    Installing : kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch 46/62
    Installing : flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch 47/62
    Installing : spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch 48/62
    Updating : perl-Compress-Zlib-2.021-144.el6.x86_64 49/62
    Updating : db4-utils-4.7.25-22.el6.x86_64 50/62
    Cleanup : perl-Compress-Zlib-2.021-141.el6_7.1.x86_64 51/62
    Cleanup : perl-IO-Compress-Zlib-2.021-141.el6_7.1.x86_64 52/62
    Cleanup : perl-IO-Compress-Base-2.021-141.el6_7.1.x86_64 53/62
    Cleanup : 1:perl-Compress-Raw-Zlib-2.021-141.el6_7.1.x86_64 54/62
    Cleanup : 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64 55/62
    Cleanup : 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64 56/62
    Cleanup : 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64 57/62
    Cleanup : 3:perl-version-0.77-141.el6_7.1.x86_64 58/62
    Cleanup : 4:perl-libs-5.10.1-141.el6_7.1.x86_64 59/62
    Cleanup : 4:perl-5.10.1-141.el6_7.1.x86_64 60/62
    Cleanup : db4-utils-4.7.25-20.el6_7.x86_64 61/62
    Cleanup : db4-4.7.25-20.el6_7.x86_64 62/62
    Verifying : solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch 1/62
    Verifying : spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch 2/62
    Verifying : zookeeper-3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6.x86_64 3/62
    Verifying : perl-Compress-Zlib-2.021-144.el6.x86_64 4/62
    Verifying : 4:perl-5.10.1-144.el6.x86_64 5/62
    Verifying : at-3.1.10-49.el6.x86_64 6/62
    Verifying : perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64 7/62
    Verifying : redhat-lsb-core-4.0-7.el6.centos.x86_64 8/62
    Verifying : pax-3.4-10.1.el6.x86_64 9/62
    Verifying : perl-Test-Harness-3.17-144.el6.x86_64 10/62
    Verifying : db4-utils-4.7.25-22.el6.x86_64 11/62
    Verifying : 1:perl-ExtUtils-ParseXS-2.2003.0-144.el6.x86_64 12/62
    Verifying : flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch 13/62
    Verifying : hadoop-0.20-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 14/62
    Verifying : 4:perl-libs-5.10.1-144.el6.x86_64 15/62
    Verifying : perl-CGI-3.51-144.el6.x86_64 16/62
    Verifying : bc-1.06.95-1.el6.x86_64 17/62
    Verifying : bigtop-jsvc-0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6.x86_64 18/62
    Verifying : parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch 19/62
    Verifying : hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 20/62
    Verifying : hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 21/62
    Verifying : hadoop-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 22/62
    Verifying : bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 23/62
    Verifying : 4:perl-devel-5.10.1-144.el6.x86_64 24/62
    Verifying : 1:perl-Pod-Simple-3.13-144.el6.x86_64 25/62
    Verifying : 1:perl-Pod-Escapes-1.04-144.el6.x86_64 26/62
    Verifying : bigtop-tomcat-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 27/62
    Verifying : parquet-format-2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6.noarch 28/62
    Verifying : time-1.7-38.el6.x86_64 29/62
    Verifying : avro-libs-1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6.noarch 30/62
    Verifying : 1:perl-Compress-Raw-Zlib-2.021-144.el6.x86_64 31/62
    Verifying : sentry-1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6.noarch 32/62
    Verifying : gdbm-devel-1.8.0-39.el6.x86_64 33/62
    Verifying : db4-cxx-4.7.25-22.el6.x86_64 34/62
    Verifying : man-1.6f-39.el6.x86_64 35/62
    Verifying : db4-4.7.25-22.el6.x86_64 36/62
    Verifying : db4-devel-4.7.25-22.el6.x86_64 37/62
    Verifying : 1:perl-Module-Pluggable-3.90-144.el6.x86_64 38/62
    Verifying : mailx-12.4-8.el6_6.x86_64 39/62
    Verifying : 3:perl-version-0.77-144.el6.x86_64 40/62
    Verifying : ed-1.1-3.3.el6.x86_64 41/62
    Verifying : hive-jdbc-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 42/62
    Verifying : hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 43/62
    Verifying : perl-IO-Compress-Zlib-2.021-144.el6.x86_64 44/62
    Verifying : perl-Test-Simple-0.92-144.el6.x86_64 45/62
    Verifying : kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch 46/62
    Verifying : hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 47/62
    Verifying : hadoop-yarn-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 48/62
    Verifying : perl-IO-Compress-Base-2.021-144.el6.x86_64 49/62
    Verifying : nc-1.84-24.el6.x86_64 50/62
    Verifying : db4-utils-4.7.25-20.el6_7.x86_64 51/62
    Verifying : perl-IO-Compress-Zlib-2.021-141.el6_7.1.x86_64 52/62
    Verifying : 4:perl-5.10.1-141.el6_7.1.x86_64 53/62
    Verifying : 1:perl-Compress-Raw-Zlib-2.021-141.el6_7.1.x86_64 54/62
    Verifying : perl-IO-Compress-Base-2.021-141.el6_7.1.x86_64 55/62
    Verifying : 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64 56/62
    Verifying : 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64 57/62
    Verifying : db4-4.7.25-20.el6_7.x86_64 58/62
    Verifying : 3:perl-version-0.77-141.el6_7.1.x86_64 59/62
    Verifying : 4:perl-libs-5.10.1-141.el6_7.1.x86_64 60/62
    Verifying : 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64 61/62
    Verifying : perl-Compress-Zlib-2.021-141.el6_7.1.x86_64 62/62 Installed:
    spark-core.noarch 0:1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6 Dependency Installed:
    at.x86_64 0:3.1.10-49.el6 avro-libs.noarch 0:1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6
    bc.x86_64 0:1.06.95-1.el6 bigtop-jsvc.x86_64 0:0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6
    bigtop-tomcat.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 bigtop-utils.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6
    db4-cxx.x86_64 0:4.7.25-22.el6 db4-devel.x86_64 0:4.7.25-22.el6
    ed.x86_64 0:1.1-3.3.el6 flume-ng.noarch 0:1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6
    gdbm-devel.x86_64 0:1.8.0-39.el6 hadoop.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6
    hadoop-0.20-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hadoop-client.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6
    hadoop-hdfs.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hadoop-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6
    hadoop-yarn.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hive.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6
    hive-jdbc.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 kite.noarch 0:1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6
    mailx.x86_64 0:12.4-8.el6_6 man.x86_64 0:1.6f-39.el6
    nc.x86_64 0:1.84-24.el6 parquet.noarch 0:1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6
    parquet-format.noarch 0:2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6 pax.x86_64 0:3.4-10.1.el6
    perl-CGI.x86_64 0:3.51-144.el6 perl-ExtUtils-MakeMaker.x86_64 0:6.55-144.el6
    perl-ExtUtils-ParseXS.x86_64 1:2.2003.0-144.el6 perl-Test-Harness.x86_64 0:3.17-144.el6
    perl-Test-Simple.x86_64 0:0.92-144.el6 perl-devel.x86_64 4:5.10.1-144.el6
    redhat-lsb-core.x86_64 0:4.0-7.el6.centos sentry.noarch 0:1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6
    solr.noarch 0:4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6 time.x86_64 0:1.7-38.el6
    zookeeper.x86_64 0:3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6 Dependency Updated:
    db4.x86_64 0:4.7.25-22.el6 db4-utils.x86_64 0:4.7.25-22.el6 perl.x86_64 4:5.10.1-144.el6
    perl-Compress-Raw-Zlib.x86_64 1:2.021-144.el6 perl-Compress-Zlib.x86_64 0:2.021-144.el6 perl-IO-Compress-Base.x86_64 0:2.021-144.el6
    perl-IO-Compress-Zlib.x86_64 0:2.021-144.el6 perl-Module-Pluggable.x86_64 1:3.90-144.el6 perl-Pod-Escapes.x86_64 1:1.04-144.el6
    perl-Pod-Simple.x86_64 1:3.13-144.el6 perl-libs.x86_64 4:5.10.1-144.el6 perl-version.x86_64 3:0.77-144.el6 Complete!
    [root@elephant yum.repos.d]#

    yum install spark-core

  6. 安裝 cloudera-manager 

    [root@elephant yum.repos.d]# yum install cloudera-manager-server
    Loaded plugins: fastestmirror, refresh-packagekit
    Setting up Install Process
    Loading mirror speeds from cached hostfile
    * base: centos.01link.hk
    * extras: centos.01link.hk
    * updates: centos.01link.hk
    Resolving Dependencies
    --> Running transaction check
    ---> Package cloudera-manager-server.x86_64 0:5.9.0-1.cm590.p0.249.el6 will be installed
    --> Processing Dependency: cloudera-manager-daemons = 5.9.0 for package: cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64
    --> Running transaction check
    ---> Package cloudera-manager-daemons.x86_64 0:5.9.0-1.cm590.p0.249.el6 will be installed
    --> Finished Dependency ResolutionDependencies Resolved==============================================================================================================================================================
    Package Arch Version Repository Size
    ==============================================================================================================================================================
    Installing:
    cloudera-manager-server x86_64 5.9.0-1.cm590.p0.249.el6 cloudera-manager 8.3 k
    Installing for dependencies:
    cloudera-manager-daemons x86_64 5.9.0-1.cm590.p0.249.el6 cloudera-manager 528 MTransaction Summary
    ==============================================================================================================================================================
    Install 2 Package(s)Total download size: 528 M
    Installed size: 679 M
    Is this ok [y/N]: y
    Downloading Packages:
    (1/2): cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64.rpm | 528 MB 00:15
    (2/2): cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64.rpm | 8.3 kB 00:00
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total 34 MB/s | 528 MB 00:15
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
    Installing : cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64 1/2
    Installing : cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64 2/2
    Verifying : cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64 1/2
    Verifying : cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64 2/2 Installed:
    cloudera-manager-server.x86_64 0:5.9.0-1.cm590.p0.249.el6 Dependency Installed:
    cloudera-manager-daemons.x86_64 0:5.9.0-1.cm590.p0.249.el6 Complete!
    [root@elephant yum.repos.d]#

    yum install cloudera-manager-server

  7. 在 monkey 主机下安装 cloudera-agent 

    yum install cloudera-manager-agent

    cloudera-manager-agent

  8. 在 elephant 主机下安装 cloudera-server 和 cloudera-manager-server-db

    yum install cloudera-manager-server
    yum install cloudera-manager-server-db-

    cloudera-manager-server

  9. 完成后可以查看安装了什么程序

    [root@elephant enmoedu]# rpm -qa | grep cloudera
    cloudera-manager-server-5.9.-.cm590.p0..el6.x86_64
    cloudera-manager-daemons-5.9.-.cm590.p0..el6.x86_64
    cloudera-manager-server-db--5.9.-.cm590.p0..el6.x86_64
    cloudera-manager-agent-5.9.-.cm590.p0..el6.x86_64[root@monkey ~]# rpm -qa | grep cloudera
    cloudera-manager-daemons-5.9.-.cm590.p0..el6.x86_64
    cloudera-manager-agent-5.9.-.cm590.p0..el6.x86_64

    rpm -qa | grep cloudera

    [root@elephant yum.repos.d]# chkconfig | grep cloudera
    cloudera-scm-agent :off :off :off :on :on :on :off
    cloudera-scm-server :off :off :off :on :on :on :off
    cloudera-scm-server-db :off :off :off :on :on :on :off

    chkconfig | grep cloudera

  10. 安装 Java 并设置环境变量

    [root@elephant x86_64]# cd /var/www/html/cloudera-manager5./RPMS/x86_64
    [root@elephant x86_64]# yum install oracle-j2sdk1.
    Failed to set locale, defaulting to C
    Loaded plugins: fastestmirror, refresh-packagekit
    Setting up Install Process
    Examining oracle-j2sdk1.-1.7.+update67-.x86_64.rpm: oracle-j2sdk1.-1.7.+update67-.x86_64
    Marking oracle-j2sdk1.-1.7.+update67-.x86_64.rpm to be installed
    Loading mirror speeds from cached hostfile
    * base: ftp.cuhk.edu.hk
    * extras: ftp.cuhk.edu.hk
    * updates: ftp.cuhk.edu.hk
    Resolving Dependencies
    --> Running transaction check
    ---> Package oracle-j2sdk1..x86_64 :1.7.+update67- will be installed
    --> Finished Dependency ResolutionDependencies Resolved==============================================================================================================================================================
    Package Arch Version Repository Size
    ==============================================================================================================================================================
    Installing:
    oracle-j2sdk1. x86_64 1.7.+update67- /oracle-j2sdk1.-1.7.+update67-.x86_64 MTransaction Summary
    ==============================================================================================================================================================
    Install Package(s)Total size: M
    Installed size: M
    Is this ok [y/N]: y
    Downloading Packages:
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
    Installing : oracle-j2sdk1.-1.7.+update67-.x86_64 /
    Verifying : oracle-j2sdk1.-1.7.+update67-.x86_64 / Installed:
    oracle-j2sdk1..x86_64 :1.7.+update67- Complete!
    [root@elephant x86_64]# java -version
    java version "1.7.0_67"
    Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
    Java HotSpot(TM) -Bit Server VM (build 24.65-b04, mixed mode)

    yum install oracle-j2sdk1.7

    echo -e 'export JAVA_HOME=/usr/java/default' >> /etc/profile
    echo -e 'export PATH=$JAVA_HOME/bin:$PATH'>> /etc/profile
    alternatives --install /usr/bin/java java /usr/java/jdk1..0_67-cloudera/bin/java
    alternatives --set java /usr/java/jdk1..0_67-cloudera/bin/java
    alternatives --install /usr/bin/javac javac /usr/java/jdk1..0_67-cloudera/bin/javac
    alternatives --set javac /usr/java/jdk1..0_67-cloudera/bin/javac
    source /etc/profile

    安装 java 并设置环境变量

启动 ClouderManager

启动 Cloudera DB 并登录 Clouder Manager 去安装所需要的应用软件

[root@elephant cloudera-manager5.]# service cloudera-scm-server-db status
pg_ctl: no server running
[root@elephant cloudera-manager5.]# service cloudera-scm-server-db start
The files belonging to this database system will be owned by user "cloudera-scm".
This user must also own the server process.The database cluster will be initialized with locale en_US.UTF8.
The default text search configuration will be set to "english".fixing permissions on existing directory /var/lib/cloudera-scm-server-db/data ... ok
creating subdirectories ... ok
selecting default max_connections ...
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in /var/lib/cloudera-scm-server-db/data/base/ ... ok
initializing pg_authid ... ok
setting password ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... okSuccess. You can now start the database server using: postgres -D /var/lib/cloudera-scm-server-db/data
or
pg_ctl -D /var/lib/cloudera-scm-server-db/data -l logfile startAdding configs
Creating SCM configuration file: /etc/cloudera-scm-server/db.properties
waiting for server to start.... done
server started
CREATE ROLE
CREATE DATABASE
Created db properties file /etc/cloudera-scm-server/db.properties
Creating DB amon for role ACTIVITYMONITOR
CREATE ROLE
CREATE DATABASE
Created DB for role ACTIVITYMONITOR
Creating DB rman for role REPORTSMANAGER
CREATE ROLE
CREATE DATABASE
Created DB for role REPORTSMANAGER
Creating DB nav for role NAVIGATOR
CREATE ROLE
CREATE DATABASE
Created DB for role NAVIGATOR
Creating DB navms for role NAVIGATORMETASERVER
CREATE ROLE
CREATE DATABASE
Created DB for role NAVIGATORMETASERVER
Enabled remote connections
waiting for server to shut down.... done
server stopped
DB initialization done.
waiting for server to start.... done
server started
[root@elephant cloudera-manager5.]#

service cloudera-scm-server-db start

[root@elephant cloudera-manager5.]# service cloudera-scm-server start
Starting cloudera-scm-server: [OK]

service cloudera-scm-server start

查看Cloudera DB 的状态

[root@elephant yum.repos.d]# service cloudera-scm-server-db status
pg_ctl: server is running (PID: )
/usr/bin/postgres "-D" "/var/lib/cloudera-scm-server-db/data"

查看Cloudera DB 的状态

登入 Cloudera Manager

  1. 登入 ClouderaManager 然后输入用户名和密码：admin/ admin

    http://elephant:7180

  2. 选择版本

  3. 选择安装的主机名称

  4. 选择 yum 的安装源

    http://elephant/cloudera-cdh5.9
    http://elephant/cloudera-manager5.9

  5. 安装中

  6. 检查系统


  7. 检查 DB
  8. 安装并启动所有服务
  9. 然后登入 Cloudera Manager 

问题

  1. cloudera-scm-server 启动不了，请检查 clouder 的日志 /var/log/cloudera-scm-server/cloudera-scm-server.out

    [root@elephant cloudera-manager5.]# service cloudera-scm-server start
    Starting cloudera-scm-server: [FAILED]
    [root@elephant log]# cd /var/log/cloudera-scm-server/
    [root@elephant cloudera-scm-server]# ls
    cloudera-scm-server.out db.log
    [root@elephant cloudera-scm-server]# cat cloudera-scm-server.out
    +======================================================================+
    | Error: JAVA_HOME is not set and Java could not be found |
    +----------------------------------------------------------------------+
    | Please download the latest Oracle JDK from the Oracle Java web site |
    | > http://www.oracle.com/technetwork/java/javase/index.html < |
    | |
    | Cloudera Manager requires Java 1.6 or later. |
    | NOTE: This script will find Oracle Java whether you install using |
    | the binary or the RPM based installer. |
    +======================================================================+

    /var/log/cloudera-scm-server/cloudera-scm-server.out

  2. check the log file: tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

    [root@elephant enmoedu]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
    -- ::, INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
    -- ::, INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
    -- ::, INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
    -- ::, INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties
    -- ::, INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.properties
    -- ::, INFO main:com.cloudera.server.cmf.components.EmbeddedDbManager: Reading embedded database password from /var/lib/cloudera-scm-server-db/data/generated_password.txt
    -- ::, INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.properties
    -- ::, INFO main:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check will use thread pool of size
    -- ::, INFO main:com.cloudera.cmf.cdhclient.util.CDHUrlClassLoader: Detected that this program is running in a JAVA 1.7.0_67 JVM. CDH5 jars will be loaded from:lib/cdh5
    -- ::, INFO main:com.cloudera.cmf.cdhclient.CdhExecutorFactory: Adding options to bad options list:

    tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

  3. 可能会遇到 Host Service is not running 信息，此时你需要检查你有没有启动 Cloudera Manager Service

  4. 启动 Host Monitor 和 Service Monitor
  5. 启动 Cluster
  6. Request to the Service Monitor failed. This may cause slow page responses. View the status of the Service Monitor.

    [root@elephant cloudera-scm-server]# cd /var/log/cloudera-scm-firehose/
    [root@elephant cloudera-scm-firehose]# ls
    mgmt-cmf-mgmt-HOSTMONITOR-elephant.log.out mgmt-cmf-mgmt-SERVICEMONITOR-elephant.log.out stacks

    /var/log/cloudera-scm-firehose/


    https://community.cloudera.com/t5/Cloudera-Manager-Installation/Request-to-the-Host-Monitor-failed-This-may-cause-slow-page/td-p/24126

  7. stopping the server cloudera-scm-server

    [root@elephant enmoedu]# service cloudera-scm-server stop
    Stopping cloudera-scm-server: [ OK ]
    [root@elephant enmoedu]# service cloudera-scm-server-db stop
    waiting for server to shut down.... done
    server stopped

    service cloudera-scm-server stop

  8. 遇上这个错误请检查是不是两个主机都启动了 httpd 服务

    [root@monkey ~]# service httpd status
    httpd is stopped
    [root@monkey ~]# service httpd start
    Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.16.201.156 for ServerName
    [ OK ]
    [root@monkey ~]# service httpd status
    httpd (pid ) is running...

    service httpd status

# cloudera master server
alias elephant='ssh -i "~/.ssh/key.pem" centos@xxx.xxx.xxx'
alias monkey='ssh -i "~/.ssh/key.pem" centos@xxx.xxx.xxx'
alias tiger='ssh -i "~/.ssh/key.pem" centos@xxx.xxx.xxx'scp -i ~/.ssh/key.pem training_materials.zip centos@xxx.xxx.xxx:/home/centos#AWS centos os 6
sudo yum update -y
sudo yum install vim -y
sudo yum install wget -y
sudo yum install gcc -y
sudo yum install unzip -y
echo "done"cat << EOF > ~/.vimrc
set number
syntax enable
set ts=4
set expandtab
set cursorline
set shiftwidth=4
EOFcat << EOF >> ~/.bashrc
export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera/
export CLASSPATH=/usr/java/jdk1.7.0_67-cloudera/jre/
export PYTHON_HOME=/usr/src/Python-3.4.5alias python3=python3.4
alias pip3=pip3.4
PATH=/usr/local/bin:$JAVA_HOME/bin:$PYTHON_HOME/bin:$PATH:$HOME/bin
EOFcat << EOF > /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6xxx.xxx.xxx elephant
xxx.xxx.xxx monkey
xxx.xxx.xxx tiger
EOF# update hostname
cat << EOF > /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=elephant
NETWORKING_IPV6=no
NOZEROCONF=yes
HOSTS=elephant
EOFhostname elephant# update hostname
cat << EOF > /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=monkey
NETWORKING_IPV6=no
NOZEROCONF=yes
HOSTS=monkey
EOFhostname monkey# update hostname
cat << EOF > /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=tiger
NETWORKING_IPV6=no
NOZEROCONF=yes
HOSTS=tiger
EOFhostname tigeruseradd enmoedu
passwd enmoeducat << EOF > /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
EOFservice iptables stop
chkconfig iptables offsudo setenforce 0
sestatus # check the selinux
#update hosts file
vim /etc/ssh/sshd_config# update a few confguration to yes
RSAAuthentication yes
PubkeyAuthentication yes
PermitRootLogin yes
PasswordAuthentication yes# restart the sshd service
sudo systemctl restart sshd
sudo service sshd restart
sudo systemctl stop iptables.servicessh-keygen -f $HOME/.ssh/id_rsa -t rsa -N ''
# generate ssh key
su enmoedu # switch to enmoedu to generate the ssh key
ssh-keygen -f $HOME/.ssh/id_rsa -t rsa -N ''
exit# switch back to root
cat /home/enmoedu/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys
cat /home/enmoedu/.ssh/id_rsa.pub >> /home/enmoedu/.ssh/authorized_keys
cat /root/.ssh/id_rsa.pub >> /home/enmoedu/.ssh/authorized_keys
chown enmoedu:enmoedu /home/enmoedu/.ssh/authorized_keyscat << EOF >> /etc/sudoers.d/90-cloud-init-users
root ALL=(ALL:ALL) ALL
%admin ALL=(ALL) ALL
%sudo ALL=(ALL:ALL) ALL
enmoedu ALL=(ALL)NOPASSWD:ALL
EOFcat << EOF >> /etc/sudoers
root ALL=(ALL:ALL) ALL
%admin ALL=(ALL) ALL
%sudo ALL=(ALL:ALL) ALL
enmoedu ALL=(ALL)NOPASSWD:ALL
EOF# edit the vm swswappiness
cat << EOF > /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
vm.swappiness=10
EOFcat << EOF > /etc/rc.local
#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.touch /var/lock/subsys/local
"echo never > /sys/kernel/mm/transparent_hugepage/defrag"
"echo never > /sys/kernel/mm/transparent_hugepage/enabled"
EOFcd ~/
wget http://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin
chmod +x cloudera-manager-installer.bin
sudo ./cloudera-manager-installer.bin# create a enmoedu folder in hadoop
sudo -u hdfs hadoop fs -mkdir /user/enmoedu
sudo -u hdfs hadoop fs -chmod +w /user/enmoedu
sudo -u hdfs hadoop fs -chown enmoedu:enmoedu /user/enmoedu# download the python source code
cd /usr/src
wget https://www.python.org/ftp/python/3.4.5/Python-3.4.5.tgz
tar xzf Python-3.4.5.tgz
cd Python-3.4.5
./configure
make altinstall
rm Python-3.4.5.tgz
python3 -Vpip3 install numpy
pip3 install matplotlib
pip3 install kazooreboot -h now# upload the cloudera training material to Serversudo -s
sudo mkdir /home/enmoedu/src
sudo chown enmoedu:enmoedu /home/enmoedu/src
sudo mv training_materials.zip /home/enmoedu/srcsu enmoedu
unzip training_materials.zip
mv training_materials/ ../
sudo chown enmoedu:enmoedu training_materials.zip# Set install Mysql (switch to monkey server)
rpm -Uvh http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
yum install mysql-community-server
/usr/bin/systemctl enable mysqld
reboot -h now# download MySQL JDBC connector
cd ~
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.43.tar.gz
tar zxvf mysql-connector-java-5.1.43.tar.gz
mkdir /usr/share/java/
cd /usr/share/java/
cp ~/mysql-connector-java-5.1.43/mysql-connector-java-5.1.43-bin.jar ./
mv mysql-connector-java-5.1.43-bin.jar mysql-connector-java.jar# Set Up Hive
mysql -u root
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';
GRANT ALL ON *.* TO 'hive'@'%';
FLUSH PRIVILEGES;
EXIT;mysql -h monkey -u hive -p# Connect to Beeline
beeline -u jdbc:hive2://monkey:10000
create database testdb;
show databases;
use testdb;
show tables;
create table testdb.test_tb1(id INT, name STRING);
drop table testdb.test_tb1;
insert into testdb.test_tb1 values (1, "Alex");

AWS CentOS Notes

参考资料

[1] Cloudera Manager 5 和 CDH5 本地（离线）安装指南