正文
使用Cloudera Manager部署Kafka消息队列
提示：扫一扫查出行【扫一扫了解最新限行尾号】
复制提示
使用Cloudera Manager部署Kafka消息队列
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.下载需要安装的Kafka版本
1>.查看 Cloudera Distribution of Apache Kafka（简称CDK）版本
CDK版本链接地址：https://www.cloudera.com/documentation/kafka/latest/topics/kafka_packaging.html#concept_fzg_phl_br
2>.查看CDK的下载地址
CDK下载链接地址：http://archive.cloudera.com/kafka/parcels/
3>.查看CDK的Custom Service Descriptor（简称CSD）所依赖的jar包下载地址
CSD链接下载地址：http://archive.cloudera.com/csds/
4>.下载CDK所依赖的jar包至Cloudera Manager Server的服务器上指定的csd路径中
[root@node101.yinzhengjie.org.cn ~]# yum -y install wget
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink | 4.3 kB ::
* base: mirrors.aliyun.com
* epel: mirrors.tuna.tsinghua.edu.cn
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
base | 3.6 kB ::
epel | 5.3 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
(/): epel/x86_64/updateinfo | kB ::
epel/x86_64/primary_db FAILED
https://ftp.yz.yamagata-u.ac.jp/pub/linux/fedora-projects/epel/7/x86_64/repodata/fff4c94b92f9dbcf982145b378cc71c45b906f382fbb54ba5d819c5ef234f32d-primary.sqlite.bz2: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below wiki article https://wiki.centos.org/yum-errors If above article doesn't help to resolve this issue please use https://bugs.centos.org/. (/): epel/x86_64/primary_db | 6.7 MB ::
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 :1.14-.el7_6. will be installed
--> Finished Dependency Resolution Dependencies Resolved ==============================================================================================================================================================================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================================================================================================================================================================
Installing:
wget x86_64 1.14-.el7_6. updates k Transaction Summary
==============================================================================================================================================================================================================================================================================
Install Package Total download size: k
Installed size: 2.0 M
Downloading packages:
wget-1.14-.el7_6..x86_64.rpm | kB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : wget-1.14-.el7_6..x86_64 /
Verifying : wget-1.14-.el7_6..x86_64 / Installed:
wget.x86_64 :1.14-.el7_6. Complete!
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# yum -y install wget #安装下载工具
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/csds/kafka/KAFKA-1.2.0.jar
---- ::-- http://archive.cloudera.com/csds/kafka/KAFKA-1.2.0.jar
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.228.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.228.167|:... connected.
HTTP request sent, awaiting response... OK
Length: (.5K) [application/java-archive]
Saving to: ‘KAFKA-1.2..jar’ %[============================================================================================================================================================>] , .4KB/s in .4s -- :: (12.4 KB/s) - ‘KAFKA-1.2..jar’ saved [/] [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/csds/kafka/KAFKA-1.2.0.jar #下载kafka的依赖包
[root@node101.yinzhengjie.org.cn ~]# mkdir /opt/cloudera/csd && cd /opt/cloudera/csd
[root@node101.yinzhengjie.org.cn /opt/cloudera/csd]#
[root@node101.yinzhengjie.org.cn /opt/cloudera/csd]#
[root@node101.yinzhengjie.org.cn /opt/cloudera/csd]# mv ~/KAFKA-1.2..jar ./
[root@node101.yinzhengjie.org.cn /opt/cloudera/csd]#
[root@node101.yinzhengjie.org.cn /opt/cloudera/csd]# ll
total
-rw-r--r-- root root Jun : KAFKA-1.2..jar
[root@node101.yinzhengjie.org.cn /opt/cloudera/csd]#
[root@node101.yinzhengjie.org.cn /opt/cloudera/csd]#
5>.下载CDK的Parcels包 至Cloudera Manager Server的服务器上指定路径
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel
---- ::-- http://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.108.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.108.167|:... connected.
HTTP request sent, awaiting response... OK
Length: (82M) [binary/octet-stream]
Saving to: ‘KAFKA-4.0.-1.4.0.0.p0.-el7.parcel’ %[============================================================================================================================================================>] ,, .6KB/s in 12m 24s -- :: ( KB/s) - ‘KAFKA-4.0.-1.4.0.0.p0.-el7.parcel’ saved [/] [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel.sha1
---- ::-- http://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel.sha1
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.228.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.228.167|:... connected.
HTTP request sent, awaiting response... OK
Length: [binary/octet-stream]
Saving to: ‘KAFKA-4.0.-1.4.0.0.p0.-el7.parcel.sha1’ %[============================================================================================================================================================>] --.-K/s in 0s -- :: (2.88 MB/s) - ‘KAFKA-4.0.-1.4.0.0.p0.-el7.parcel.sha1’ saved [/] [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.0.0-1.4.0.0.p0.1-el7.parcel.sha1
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/kafka/parcels/latest/manifest.json
---- ::-- http://archive.cloudera.com/kafka/parcels/latest/manifest.json
Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.108.167
Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.108.167|:... connected.
HTTP request sent, awaiting response... OK
Length: (.1K) [application/json]
Saving to: ‘manifest.json’ %[============================================================================================================================================================>] , .7KB/s in .3s -- :: (19.7 KB/s) - ‘manifest.json’ saved [/] [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# wget http://archive.cloudera.com/kafka/parcels/latest/manifest.json
[root@node101.yinzhengjie.org.cn ~]# ll #下载的3个文件
total
-rw-r--r-- root root Apr : KAFKA-4.0.-1.4.0.0.p0.-el7.parcel
-rw-r--r-- root root Apr : KAFKA-4.0.-1.4.0.0.p0.-el7.parcel.sha1
-rw-r--r-- root root Apr : manifest.json
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /opt/cloudera/parcel-repo/ #现有的parcel源
total
-rw-r--r-- root root Jan : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Jan : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
-rw-r----- root root Jun : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.torrent
-rw-r--r-- root root Jan : manifest.json
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# mv KAFKA-4.0.-1.4.0.0.p0.-el7.parcel /opt/cloudera/parcel-repo/ #将parcel包移动过来
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# mv KAFKA-4.0.-1.4.0.0.p0.-el7.parcel.sha1 /opt/cloudera/parcel-repo/KAFKA-4.0.-1.4.0.0.p0.-el7.parcel.sha #移动过来我们也对其进行了改名操作
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /opt/cloudera/parcel-repo/
total
-rw-r--r-- root root Jan : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Jan : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
-rw-r----- root root Jun : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.torrent
-rw-r--r-- root root Apr : KAFKA-4.0.-1.4.0.0.p0.-el7.parcel
-rw-r--r-- root root Apr : KAFKA-4.0.-1.4.0.0.p0.-el7.parcel.sha
-rw-r--r-- root root Jan : manifest.json
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll
total
-rw-r--r-- root root Apr : manifest.json
[root@node101.yinzhengjie.org.cn ~]#
6>.对比下载的CDK版本和官网的hash是否一致（由于我是从官方下载的，自然是一致的）
二.在Cloudera Manager WebUI中激活Kafka源
1>.点击"主机"，选择"Parcel"
2>.找到对应的CDK，点击"分配"
3>.等待CDK分发完成（大约30-60秒左右）
4>.点击"激活"
5>.点击"确定"
6>.等待激活完成（大约30-60秒）
三.添加CDK服务
1>.点击"添加服务"
2>.选择kafka服务（我们已经把CDK部署且激活成功啦）
3>.自定义kafka角色分配
4>.自定义kafka相关参数（如果遇到提示的错误，我们可以修改log.dirs参数）
5>.点击"继续"
6>.等待安装完毕,点击"继续"（如果集群在启动时出错的话，可以查看角色日志进行排错处理！）
7>.成功将kafka集群加入CM管理
8>.我们可以进入kafka服务的管理界面
四.测试kafka集群是否可用
1>.创建一个topic
[root@node101.yinzhengjie.org.cn ~]# kafka-topics --zookeeper node102.yinzhengjie.org.cn: --create --partitions --replication-factor --topic yinzhengjie-kafka
// :: INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Initializing a new session to node102.yinzhengjie.org.cn:.
// :: INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.-cdh5.14.2--, built on // : GMT
// :: INFO zookeeper.ZooKeeper: Client environment:host.name=node101.yinzhengjie.org.cn
// :: INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_201
// :: INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
// :: INFO zookeeper.ZooKeeper: Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
// :: INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/activation-1.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/activation-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/aopalliance-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/aopalliance-repackaged-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/apacheds-i18n-2.0.-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/apacheds-jdbm1-2.0.-M2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/apacheds-kerberos-codec-2.0.-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/api-asn1-api-1.0.-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/api-util-1.0.-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/argparse4j-0.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/asm-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/avro-1.7.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/caffeine-2.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/cglib-2.2.-v20090111.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/checker-qual-2.6..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-beanutils-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-beanutils-core-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-cli-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-codec-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-collections-3.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-compress-1.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-configuration-1.6.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-digester-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-el-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-httpclient-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-io-2.4.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-lang-2.6.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-lang3-3.5.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-logging-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-math3-3.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-net-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-pool2-2.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/compileScala.mapping:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-api-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-basic-auth-extension-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-file-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-json-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-runtime-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-transforms-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/error_prone_annotations-2.3..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/gson-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/guava-20.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/guice-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/guice-servlet-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-annotations-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-archives-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-auth-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-core-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-jobclient-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-shuffle-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-api-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-client-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-server-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-server-nodemanager-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hive-hcatalog-core-1.1.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hive-hcatalog-server-extensions-1.1.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hk2-api-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hk2-locator-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hk2-utils-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/htrace-core4-4.0.-incubating.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/httpclient-4.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/httpcore-4.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-annotations-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-annotations-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-core-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-core-asl-1.9.-cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-databind-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-jaxrs-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-jaxrs-base-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-jaxrs-json-provider-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-mapper-asl-1.9.-cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-module-jaxb-annotations-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-xc-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javassist-3.22.-CR2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.annotation-api-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.inject-.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.inject-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.servlet-api-3.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jaxb-api-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jaxb-api-2.3..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-client-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-common-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-container-servlet-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-guice-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-hk2-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-server-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jets3t-0.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jettison-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-6.1..cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-client-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-continuation-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-http-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-io-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-security-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-server-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-servlet-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-servlets-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-util-6.1..cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-util-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jopt-simple-5.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jsch-0.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jsp-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jsr305-3.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka_2.-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka_2.-2.1.-kafka-4.0.-sources.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-clients-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-log4j-appender-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-examples-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-scala_2.-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-test-utils-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-tools-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/libthrift-0.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/log4j-1.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/lz4-java-1.5..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/maven-artifact-3.5..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/metrics-core-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/metrics-servlet-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/netty-3.10..Final.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/osgi-resource-locator-1.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/paranamer-2.3.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/plexus-utils-3.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/protobuf-java-2.5..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/reflections-0.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/rocksdbjni-5.14..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/scala-library-2.11..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/scala-logging_2.-3.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/scala-reflect-2.11..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-binding-hive-conf-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-binding-hive-follower-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-binding-kafka-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-model-db-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-model-indexer-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-model-kafka-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-hdfs-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-policy-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-policy-indexer-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-policy-kafka-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-provider-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-provider-db-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-provider-file-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/servlet-api-2.5.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/shiro-core-1.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/slf4j-api-1.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/slf4j-api-1.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/slf4j-log4j12-1.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/snappy-java-1.1.7.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/stax-api-1.0-.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/validation-api-1.1..Final.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/xmlenc-0.52.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/xz-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/zkclient-0.10.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/zookeeper-3.4.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/zstd-jni-1.3.-.jar:/etc/kafka/conf/sentry-conf
// :: INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
// :: INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
// :: INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
// :: INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
// :: INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
// :: INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.-.el7.x86_64
// :: INFO zookeeper.ZooKeeper: Client environment:user.name=root
// :: INFO zookeeper.ZooKeeper: Client environment:user.home=/root
// :: INFO zookeeper.ZooKeeper: Client environment:user.dir=/root
// :: INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node102.yinzhengjie.org.cn: sessionTimeout= watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@5b8dfcc1
// :: INFO zookeeper.ClientCnxn: Opening socket connection to server node102.yinzhengjie.org.cn/172.30.1.102:. Will not attempt to authenticate using SASL (unknown error)
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Waiting until connected.
// :: INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.30.1.101:, server: node102.yinzhengjie.org.cn/172.30.1.102:
// :: INFO zookeeper.ClientCnxn: Session establishment complete on server node102.yinzhengjie.org.cn/172.30.1.102:, sessionid = 0x36b59951e91006e, negotiated timeout =
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Connected.
// :: INFO zk.AdminZkClient: Topic creation Map(yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ), yinzhengjie-kafka- -> ArrayBuffer(, ))
Created topic "yinzhengjie-kafka".
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closing.
// :: INFO zookeeper.ZooKeeper: Session: 0x36b59951e91006e closed
// :: INFO zookeeper.ClientCnxn: EventThread shut down
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closed.
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# kafka-topics --zookeeper node102.yinzhengjie.org.cn:2181 --create --partitions 20 --replication-factor 2 --topic yinzhengjie-kafka
[root@node101.yinzhengjie.org.cn ~]# zookeeper-client -server node102.yinzhengjie.org.cn
Connecting to node102.yinzhengjie.org.cn
-- ::, [myid:] - INFO [main:Environment@] - Client environment:zookeeper.version=3.4.-cdh5.15.1--, built on // : GMT
-- ::, [myid:] - INFO [main:Environment@] - Client environment:host.name=node101.yinzhengjie.org.cn
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.version=1.8.0_201
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.vendor=Oracle Corporation
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.class.path=/opt/cloudera/parcels/CDH-5.15.-.cdh5.15.1.p0./lib/zookeeper/bin/../build/classes:/opt/cloudera/parcels/CDH-5.15.-.cdh5.15.1.p0./lib/zookeeper/bin/../build/lib/*.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../lib/slf4j-log4j12-1.7.5.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../lib/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../lib/jline-2.11.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../zookeeper-3.4.5-cdh5.15.1.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/lib/zookeeper/bin/../src/java/lib/*.jar:/etc/zookeeper/conf::/etc/zookeeper/conf:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/zookeeper.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/zookeeper-3.4.5-cdh5.15.1.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/lib/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/lib/netty-3.10.5.Final.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/lib/log4j-1.2.16.jar:/opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/zookeeper/lib/jline-2.11.jar
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2019-06-15 14:35:17,405 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root
2019-06-15 14:35:17,406 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=node102.yinzhengjie.org.cn sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@306a30c7
Welcome to ZooKeeper!
2019-06-15 14:35:17,424 [myid:] - INFO [main-SendThread(node102.yinzhengjie.org.cn:2181):ClientCnxn$SendThread@975] - Opening socket connection to server node102.yinzhengjie.org.cn/172.30.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-06-15 14:35:17,469 [myid:] - INFO [main-SendThread(node102.yinzhengjie.org.cn:2181):ClientCnxn$SendThread@852] - Socket connection established, initiating session, client: /172.30.1.101:50182, server: node102.yinzhengjie.org.cn/172.30.1.102:2181
2019-06-15 14:35:17,477 [myid:] - INFO [main-SendThread(node102.yinzhengjie.org.cn:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server node102.yinzhengjie.org.cn/172.30.1.102:2181, sessionid = 0x36b59951e91006f, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
[zk: node102.yinzhengjie.org.cn(CONNECTED) 0] ls /brokers
[ids, topics, seqid]
[zk: node102.yinzhengjie.org.cn(CONNECTED) 1]
[zk: node102.yinzhengjie.org.cn(CONNECTED) 1] ls /brokers/topics
[yinzhengjie-kafka]
[zk: node102.yinzhengjie.org.cn(CONNECTED) 2]
[zk: node102.yinzhengjie.org.cn(CONNECTED) 2] ls /brokers/ids
[88, 86, 87]
[zk: node102.yinzhengjie.org.cn(CONNECTED) 3]
[zk: node102.yinzhengjie.org.cn(CONNECTED) 3] quit
Quitting...
2019-06-15 14:35:37,688 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x36b59951e91006f closed
2019-06-15 14:35:37,688 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@512] - EventThread shut down
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zookeeper-client -server node102.yinzhengjie.org.cn #查看zookeeper对应的topic信息
[root@node101.yinzhengjie.org.cn ~]# kafka-topics --zookeeper node104.yinzhengjie.org.cn: --list
// :: INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Initializing a new session to node104.yinzhengjie.org.cn:.
// :: INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.-cdh5.14.2--, built on // : GMT
// :: INFO zookeeper.ZooKeeper: Client environment:host.name=node101.yinzhengjie.org.cn
// :: INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_201
// :: INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
// :: INFO zookeeper.ZooKeeper: Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
// :: INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/activation-1.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/activation-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/aopalliance-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/aopalliance-repackaged-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/apacheds-i18n-2.0.-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/apacheds-jdbm1-2.0.-M2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/apacheds-kerberos-codec-2.0.-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/api-asn1-api-1.0.-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/api-util-1.0.-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/argparse4j-0.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/asm-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/avro-1.7.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/caffeine-2.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/cglib-2.2.-v20090111.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/checker-qual-2.6..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-beanutils-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-beanutils-core-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-cli-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-codec-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-collections-3.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-compress-1.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-configuration-1.6.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-digester-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-el-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-httpclient-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-io-2.4.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-lang-2.6.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-lang3-3.5.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-logging-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-math3-3.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-net-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/commons-pool2-2.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/compileScala.mapping:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-api-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-basic-auth-extension-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-file-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-json-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-runtime-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/connect-transforms-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/error_prone_annotations-2.3..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/gson-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/guava-20.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/guice-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/guice-servlet-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-annotations-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-archives-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-auth-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-core-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-jobclient-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-shuffle-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-api-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-client-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-server-common-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hadoop-yarn-server-nodemanager-2.6.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hive-hcatalog-core-1.1.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hive-hcatalog-server-extensions-1.1.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hk2-api-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hk2-locator-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/hk2-utils-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/htrace-core4-4.0.-incubating.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/httpclient-4.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/httpcore-4.4..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-annotations-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-annotations-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-core-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-core-asl-1.9.-cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-databind-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-jaxrs-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-jaxrs-base-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-jaxrs-json-provider-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-mapper-asl-1.9.-cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-module-jaxb-annotations-2.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jackson-xc-1.8..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javassist-3.22.-CR2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.annotation-api-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.inject-.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.inject-2.5.-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.servlet-api-3.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jaxb-api-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jaxb-api-2.3..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-client-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-common-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-container-servlet-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-guice-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-hk2-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jersey-server-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jets3t-0.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jettison-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-6.1..cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-client-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-continuation-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-http-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-io-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-security-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-server-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-servlet-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-servlets-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-util-6.1..cloudera..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jetty-util-9.4..v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jopt-simple-5.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jsch-0.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jsp-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/jsr305-3.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka_2.-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka_2.-2.1.-kafka-4.0.-sources.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-clients-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-log4j-appender-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-examples-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-scala_2.-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-streams-test-utils-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/kafka-tools-2.1.-kafka-4.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/libthrift-0.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/log4j-1.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/lz4-java-1.5..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/maven-artifact-3.5..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/metrics-core-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/metrics-servlet-2.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/netty-3.10..Final.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/osgi-resource-locator-1.0..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/paranamer-2.3.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/plexus-utils-3.1..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/protobuf-java-2.5..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/reflections-0.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/rocksdbjni-5.14..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/scala-library-2.11..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/scala-logging_2.-3.9..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/scala-reflect-2.11..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-binding-hive-conf-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-binding-hive-follower-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-binding-kafka-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-model-db-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-model-indexer-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-core-model-kafka-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-hdfs-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-policy-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-policy-indexer-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-policy-kafka-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-provider-common-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-provider-db-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/sentry-provider-file-1.5.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/servlet-api-2.5.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/shiro-core-1.2..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/slf4j-api-1.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/slf4j-api-1.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/slf4j-log4j12-1.7..jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/snappy-java-1.1.7.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/stax-api-1.0-.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/validation-api-1.1..Final.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/xmlenc-0.52.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/xz-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/zkclient-0.10.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/zookeeper-3.4.-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.-1.4.0.0.p0./bin/../lib/kafka/bin/../libs/zstd-jni-1.3.-.jar:/etc/kafka/conf/sentry-conf
// :: INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
// :: INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
// :: INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
// :: INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
// :: INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
// :: INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.-.el7.x86_64
// :: INFO zookeeper.ZooKeeper: Client environment:user.name=root
// :: INFO zookeeper.ZooKeeper: Client environment:user.home=/root
// :: INFO zookeeper.ZooKeeper: Client environment:user.dir=/root
// :: INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node104.yinzhengjie.org.cn: sessionTimeout= watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@67c27493
// :: INFO zookeeper.ClientCnxn: Opening socket connection to server node104.yinzhengjie.org.cn/172.30.1.104:. Will not attempt to authenticate using SASL (unknown error)
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Waiting until connected.
// :: INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.30.1.101:, server: node104.yinzhengjie.org.cn/172.30.1.104:
// :: INFO zookeeper.ClientCnxn: Session establishment complete on server node104.yinzhengjie.org.cn/172.30.1.104:, sessionid = 0x26b59951e92006d, negotiated timeout =
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Connected.
__consumer_offsets
yinzhengjie-kafka
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closing.
// :: INFO zookeeper.ZooKeeper: Session: 0x26b59951e92006d closed
// :: INFO zookeeper.ClientCnxn: EventThread shut down
// :: INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closed.
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# kafka-topics --zookeeper node104.yinzhengjie.org.cn:2181 --list #我们也可以使用kafka提供的命令行来查看topic信息哟~
2>.创建一个生产者
[root@node101.yinzhengjie.org.cn ~]# kafka-console-producer --broker-list node103.yinzhengjie.org.cn: --topic yinzhengjie-kafka
// :: INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean
// :: INFO producer.ProducerConfig: ProducerConfig values:
acks =
batch.size =
bootstrap.servers = [node103.yinzhengjie.org.cn:]
buffer.memory =
client.dns.lookup = default
client.id = console-producer
compression.type = none
connections.max.idle.ms =
delivery.timeout.ms =
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms =
max.block.ms =
max.in.flight.requests.per.connection =
max.request.size =
metadata.max.age.ms =
metric.reporters = []
metrics.num.samples =
metrics.recording.level = INFO
metrics.sample.window.ms =
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes =
reconnect.backoff.max.ms =
reconnect.backoff.ms =
request.timeout.ms =
retries =
retry.backoff.ms =
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin =
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds =
sasl.login.refresh.min.period.seconds =
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes =
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1., TLSv1., TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms =
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer // :: INFO utils.AppInfoParser: Kafka version : 2.1.-kafka-4.0.
// :: INFO utils.AppInfoParser: Kafka commitId : unknown
> I Love You! #注意，只有这一条才是我发送的消息！！！！！
// :: INFO clients.Metadata: Cluster ID: QDWDZmAUQkW-qkkZcE_OZQ
>
[root@node101.yinzhengjie.org.cn ~]# kafka-console-producer --broker-list node103.yinzhengjie.org.cn:9092 --topic yinzhengjie-kafka
3>.创建一个消费者
[root@node105.yinzhengjie.org.cn ~]# kafka-console-consumer --bootstrap-server node102.yinzhengjie.org.cn: --topic yinzhengjie-kafka --from-beginning
// :: INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean
// :: INFO consumer.ConsumerConfig: ConsumerConfig values:
auto.commit.interval.ms =
auto.offset.reset = earliest
bootstrap.servers = [node102.yinzhengjie.org.cn:]
check.crcs = true
client.dns.lookup = default
client.id =
connections.max.idle.ms =
default.api.timeout.ms =
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes =
fetch.max.wait.ms =
fetch.min.bytes =
group.id = console-consumer-
heartbeat.interval.ms =
interceptor.classes = []
internal.leave.group.on.close = true
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes =
max.poll.interval.ms =
max.poll.records =
metadata.max.age.ms =
metric.reporters = []
metrics.num.samples =
metrics.recording.level = INFO
metrics.sample.window.ms =
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes =
reconnect.backoff.max.ms =
reconnect.backoff.ms =
request.timeout.ms =
retry.backoff.ms =
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin =
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds =
sasl.login.refresh.min.period.seconds =
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes =
session.timeout.ms =
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1., TLSv1., TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer // :: INFO utils.AppInfoParser: Kafka version : 2.1.-kafka-4.0.
// :: INFO utils.AppInfoParser: Kafka commitId : unknown
// :: INFO clients.Metadata: Cluster ID: QDWDZmAUQkW-qkkZcE_OZQ
// :: INFO internals.AbstractCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Discovered group coordinator node104.yinzhengjie.org.cn: (id: rack: null)
// :: INFO internals.ConsumerCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Revoking previously assigned partitions []
// :: INFO internals.AbstractCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] (Re-)joining group
// :: INFO internals.AbstractCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Successfully joined group with generation
// :: INFO internals.ConsumerCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Setting newly assigned partitions [yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-, yinzhengjie-kafka-]
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition yinzhengjie-kafka- to offset .
I Love You! #注意，这一条消息才是我要接受的哟~
[root@node105.yinzhengjie.org.cn ~]# kafka-console-consumer --bootstrap-server node102.yinzhengjie.org.cn:9092 --topic yinzhengjie-kafka --from-beginning
