安装Zookeeper和Kafka集群 - 南瓜慢说
source link: https://www.cnblogs.com/larrydpk/p/17325843.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
安装Zookeeper和Kafka集群
本文介绍如何安装Zookeeper和Kafka集群。为了方便,介绍的是在一台服务器上的安装,实际应该安装在多台服务器上,但步骤是一样的。
安装Zookeeper集群
下载安装包
从官网上下载安装包:
curl https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz -o apache-zookeeper-3.7.1-bin.tar.gz
tar xvf apache-zookeeper-3.7.1-bin.tar.gz
创建目录 zk1
,然后添加如下配置:
zk1/myid
:
zk1/zk.config
:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk1
clientPort=2181
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
对于zk2
和zk3
也重复同样的步骤,并修改相应的配置:
zk2/myid
:
2
zk2/zk.config
:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk2
clientPort=2182
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
zk3/myid
:
3
zk3/zk.config
:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk3
clientPort=2183
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
启动三个服务如下:
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk1/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk1/zk.config
Starting zookeeper ... STARTED
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk2/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk2/zk.config
Starting zookeeper ... STARTED
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk3/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk3/zk.config
Starting zookeeper ... STARTED
通过status
命令查看:
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk1/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk1/zk.config
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk2/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk2/zk.config
Client port found: 2182. Client address: localhost. Client SSL: false.
Mode: leader
$ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk3/zk.config
ZooKeeper JMX enabled by default
Using config: ./zk3/zk.config
Client port found: 2183. Client address: localhost. Client SSL: false.
Mode: follower
连接其中一个服务并添加数据:
$ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2181
[zk: localhost:2181(CONNECTED) 0] create /pkslow
Created /pkslow
[zk: localhost:2181(CONNECTED) 1] create /pkslow/website www.pkslow.com
Created /pkslow/website
连接另外一个服务,并查看数据,发现与之前创建的是一样的:
$ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2182
[zk: localhost:2182(CONNECTED) 1] get /pkslow/website
www.pkslow.com
目录结构如下:
安装Kafka集群
下载安装包
通过官网下载如下:
curl https://downloads.apache.org/kafka/3.4.0/kafka_2.13-3.4.0.tgz -o kafka_2.13-3.4.0.tgz
解压安装包:
tar -xzf kafka_2.13-3.4.0.tgz
Broker 1
的配置如下:
broker.id=1
port=9091
listeners=PLAINTEXT://:9091
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka1/kafka-logs
Broker 2
的配置如下:
broker.id=2
port=9092
listeners=PLAINTEXT://:9092
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka2/kafka-logs
Broker 3
的配置如下:
broker.id=3
port=9093
listeners=PLAINTEXT://:9093
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka3/kafka-logs
目录结构如下:
启动kafka
服务如下:
./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka1/server.properties
./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka2/server.properties
./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka3/server.properties
检查如测试
创建topic:
$ kafka_2.13-3.4.0/bin/kafka-topics.sh --create --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --partitions 3 --replication-factor 3
Created topic pkslow-topic.
列出topic:
$ kafka_2.13-3.4.0/bin/kafka-topics.sh --list --bootstrap-server localhost:9091,localhost:9092,localhost:9093
pkslow-topic
查看topic:
$ kafka_2.13-3.4.0/bin/kafka-topics.sh --describe --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093
Topic: pkslow-topic TopicId: 7CLy7iZeRvm8rCrn8Dw_mA PartitionCount: 3 ReplicationFactor: 3 Configs:
Topic: pkslow-topic Partition: 0 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2
Topic: pkslow-topic Partition: 1 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: pkslow-topic Partition: 2 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1
生产者发消息到brokers:
$ kafka_2.13-3.4.0/bin/kafka-console-producer.sh --broker-list localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic
>My name is Larry Deng.
>My website is www.pkslow.com.
>
消费者从brokers收消息:
$ kafka_2.13-3.4.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic --from-beginning
My name is Larry Deng.
My website is www.pkslow.com.
配置可以参考 GitHub pkslow-samples
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK