Teem Up Kafka: 4 Kauj Ruam

2024 Tus sau: John Day | [email protected]. Kawg hloov kho: 2024-01-30 09:27

Taw qhia:

Apache Kafka yog qhov qhib tau qhov ntsuas tau thiab muaj peev xwm xa xov tau zoo tsim los ntawm Apache Software Foundation sau hauv Scala. Apache Kafka tau qhia tshwj xeeb uas tsim los tso rau ib pawg los ua lub hauv paus cov ntaub ntawv qaum rau ib puag ncig loj. Nws muaj qhov kev nkag siab ntau dua piv rau lwm cov lus xa xov tshuab xws li ActiveMQ thiab RabbitMQ. Nws muaj peev xwm tswj hwm tag nrho cov ntaub ntawv ntawm lub sijhawm tiag tiag. Koj tuaj yeem xa Kafka ntawm ib tus neeg rau zaub mov Apache lossis hauv ib puag ncig faib ua pawg.

Nta:

Cov yam ntxwv dav dav ntawm Kafka yog raws li hauv qab no:

Ua raws cov lus ntawm disk uas muab sijhawm ua haujlwm tas li.

Kev nkag siab siab nrog cov txheej txheem disk uas txhawb ntau pua txhiab tus lus ib pliag.

Faib qhov system nplai yooj yim nrog tsis muaj sijhawm poob.

Txhawb nqa ntau tus neeg siv npe thiab txiav cov neeg siv khoom thaum lub sijhawm ua tsis tiav.

Phau ntawv qhia no qhia yuav ua li cas rau nruab thiab teeb tsa Apache Kafka ntawm Ubuntu 16.04 server.

Yuav tsum muaj

Ubuntu 16.04 neeg rau zaub mov.

Cov tsis siv hauv paus tus lej siv nrog cov neeg siv tshwj xeeb tsim nyog teeb tsa ntawm koj lub server.

Kauj ruam 1: Pib thiab Txhim Kho Java

1) Cia peb pib ua kom ntseeg tau tias koj li Ubuntu 16.04 neeg rau zaub mov tau ua tiav

Koj tuaj yeem hloov kho koj lub server los ntawm kev ua cov lus txib hauv qab no:-

sudo apt -tau hloov tshiab -y

sudo apt -tau hloov tshiab -y

2) Txhim kho Java

Txheeb xyuas yog tias koj lub tshuab muaj java uas twb tau teeb tsa lossis muaj java default version los ntawm cov lus txib hauv qab no:-

java -version

Txawm hais tias koj muaj java tab sis qis dua version, Koj yuav tau hloov kho nws.

Koj tuaj yeem nruab Java los ntawm:-

sudo apt-tau nruab default-jdk

LOS YOG

Koj tuaj yeem nruab Oracle JDK 8 siv Webupd8 pab pawg PPA chaw cia khoom.

Txhawm rau ntxiv cov chaw cia khoom, khiav cov lus txib hauv qab no:-

sudo ntxiv-apt-repository -y ppa: webupd8team/java

sudo apt-tau nruab oracle-java8-installer -y

Kauj ruam 2: Nruab Zookeeper

Zookeeper yog dab tsi?

Zookeeper yog qhov kev pabcuam hauv nruab nrab rau kev tswj hwm cov ntaub ntawv teeb tsa, npe, muab kev faib ua synchronization, thiab muab kev pabcuam pab pawg. Txhua yam ntawm cov kev pabcuam no tau siv hauv qee daim lossis lwm qhov los ntawm cov ntawv thov faib. Txhua lub sijhawm lawv tau ua tiav muaj ntau txoj haujlwm uas nkag mus rau kev kho cov kab thiab cov xwm txheej ntawm haiv neeg uas tsis tuaj yeem zam tau. Vim tias qhov nyuaj ntawm kev siv cov kev pabcuam no, daim ntawv thov pib feem ntau hla lawv, uas ua rau lawv nkig thaum muaj kev hloov pauv thiab nyuaj rau kev tswj hwm. Txawm hais tias ua tiav kom raug, kev siv sib txawv ntawm cov kev pabcuam no ua rau kev tswj hwm nyuaj thaum cov ntawv thov raug xa mus.

Ua ntej txhim kho Apache Kafka, koj yuav tsum muaj zookeeper muaj thiab khiav. ZooKeeper yog qhov kev pabcuam qhib rau kev tswj hwm cov ntaub ntawv teeb tsa, muab faib sib koom ua ke, npe thiab muab kev pabcuam pab pawg.

1) Los ntawm lub neej qub Zookeeper pob muaj nyob hauv Ubuntu lub chaw cia khoom qub

Koj tuaj yeem nruab nws los ntawm kev ua cov lus txib hauv qab no:-

sudo apt-tau nruab zookeeperd

Thaum qhov kev teeb tsa tiav, nws yuav pib ua daemon txiav. Los ntawm lub neej ntawd Zookeeper yuav khiav ntawm chaw nres nkoj 2181.

Koj tuaj yeem sim nws los ntawm kev ua cov lus txib hauv qab no:

netstat -ant | ib: 2181

Qhov tso tawm yuav tsum qhia koj tias chaw nres nkoj 2181 tau mloog.

Kauj Ruam 3: Nruab thiab Pib Kafka Server

Tam sim no Java thiab ZooKeeper tau teeb tsa, nws yog lub sijhawm rub tawm thiab rho tawm Kafka los ntawm Apache lub vev xaib.

1) Koj tuaj yeem siv curl lossis wget mus rub Kafka: (Kafka version 0.10.1.1)

Khiav cov lus txib hauv qab no txhawm rau rub tawm teeb tsa kafka:-

curl -O

LOS YOG

wget

2) Tsim cov npe rau Kafka

Tom ntej no, tsim cov npe rau Kafka teeb tsa:

sudo mkdir /xaiv /kafka

cd /opt /cav

3) Unzip downloaded nplaub tshev

sudo tar -zxvf /home/user_name/Downloads/kafka_2.11-0.10.1.1.tgz -C/opt/kafka/

*Hloov tus neeg siv lub npe raws li koj lub npe siv

4) Pib kafka server

Cov kauj ruam tom ntej yog pib Kafka server, koj tuaj yeem pib nws los ntawm kev khiav kafka-server-start.sh tsab ntawv nyob ntawm /opt/kafka/kafka_2.11-0.10.1.1/bin/ directory los ntawm kev siv cov lus txib hauv qab no:-

sudo /opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-server-start.sh /opt/kafka/kafka_2.11-0.10.1.1/config/server.properties

5) Txheeb xyuas yog tias Kafka Server ua haujlwm tau zoo

Tam sim no koj muaj Kafka server khiav thiab mloog ntawm chaw nres nkoj 9092.

Tam sim no, peb tuaj yeem tshawb xyuas cov chaw mloog:

ZooKeeper: 2181

- Kev: 9092

netstat -ant | grep -E ': 2181 |: 9092'

Kauj Ruam 4: Ntsuam Xyuas Koj Kafka Server

Tam sim no, nws yog lub sijhawm los txheeb xyuas Kafka server tau ua haujlwm raug.

1) Tsim lub ncauj lus tshiab

Txhawm rau sim Kafka, tsim cov ncauj lus piv txwv nrog lub npe "kuaj" hauv Apache Kafka siv cov lus txib hauv qab no:

/opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-topics.sh-tsim --topic xeem --zookeeper localhost: 2181-partitions 1 --replication-factor 1

2) Txheeb xyuas yog tias koj lub ncauj lus tau tsim ua tiav

Tam sim no, thov Zookeeper sau cov ncauj lus muaj nyob hauv Apache Kafka los ntawm kev ua cov lus txib hauv qab no:

/opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-topics.sh --list --zookeeper localhost: 2181

3) Tshaj tawm cov lus siv lub ncauj lus koj tsim

ncha "nyob zoo ntiaj teb" | /opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-console-producer.sh --broker-list localhost: 9092 --topic xeem

4) Tau txais cov lus ntawm lub ncauj lus tsim

/opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-console-consumer.sh-bootstrap-server localhost: 9092-kev xeem saum toj no-pib-pib

5) Txhawm rau xa cov ntawv siv kafka hla cov ncauj lus

kafka-console-producer.sh --broker-list localhost: 9092 testingtopic xeem