Zookeeper Kafka 安装

Zookeeper 安装

下载Zookeeper

官方下载地址https://zookeeper.apache.org/releases.html

环境搭建

1. 解压下载包，并将解压的目录重命名为zookeeper
2. 将 zookeeper/conf/zoo_sample.cfg 拷贝一份命名为 zoo.cfg
3. zoo.cfg配置

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=~/zookeeper/data
dataLogDir=~/zookeeper/logs
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

server.1=kafka-server1:2888:3888
server.2=kafka-server2:2888:3888

配置文件中的dataDir和dataLogDir最好设置在zookeeper目录下，并在集群中每台主机的这两个目录下写一个myid文件，文件内容为集群中每台主机对应的集群编号，编号与配置文件zoo.cfg最后两行对应，即kafka-server1这台主机的编号为1，kafka-server2的编号为2`。

1. 在每台主机上zookeeper/bin目录下运行./zkSever.sh start启动集群。
2. 在每台主机上zookeeper/bin目录下运行./zkServer.sh status查看状态。

   启动成功为看到集群中一台主机为leader其他主机为follower，如果启动失败，会提示not running

3. 每台主机上zookeeper/bin目录下运行./zkServer.sh stop关闭集群

kafka 安装

下载kafka

官方下载地址https://kafka.apache.org/

环境搭建

kafka依赖于zookeeper管理，因此安装kafka要先安装zookeeper并确保zookeeper安装成功，同样启动kafka要先启动zookeeper

1. 配置server.properties
server.properties中修改三个配置: breker.id设置为该主机在zookeeper集群中的编号; listeners监听该主机的9092端口，具体值在配置文件中有提示; zookeeper.connect设置为zookeeper集群，例如kakfa-server1:2181,kafka-server2:2181

2. 配置producer.properties
producer.properties中配置bootstrap.servers为kafka-server1:9092,kafka-server2:9092

3. 配置consumer.properties
consumer.properties中配置zookeeper.connect为kakfa-server1:2181,kafka-server2:2181

4. 每台主机启动kafka

   bin/kafka-server-start.sh -daemon config/server.properties

5. 每台主机关闭kafka

   bin/kafka-server-stop

zookeeper and kafka

启动集群时，先启动zookeeper，再启动kafka；关闭集群时先关闭kafka，再关闭zookeeper

建议

集群是否搭建成功关键在于有没有比较清楚的了解zookeeper和kafka，网上教程很多，照搬网上教程是大忌，要充分理解配置的意义，诸如kafka-server1``kafka-server2在集群通信问题，以及部分教程要求修改/etc/hosts的目的。没有充分理解配置，配置集群是一条漫长而迷茫的路。如果在linux中配置过一些东西，会有相当多的经验，经验也很重要。