{"id":2093,"date":"2022-02-11T08:17:19","date_gmt":"2022-02-11T08:17:19","guid":{"rendered":"https:\/\/blog.samarthya.me\/wps\/?p=2093"},"modified":"2022-02-25T06:20:13","modified_gmt":"2022-02-25T06:20:13","slug":"kafka-deploy-test","status":"publish","type":"post","link":"https:\/\/blog.samarthya.me\/wps\/2022\/02\/11\/kafka-deploy-test\/","title":{"rendered":"Kafka: Deploy & Test"},"content":{"rendered":"\n

While I continue to learn more about architecture and design of kafka<\/code>, a quickstart to deploy & test kafka is a must. <\/p>\n\n\n\n

Step 1: Add Chart Repo<\/h2>\n\n\n\n

In my local system it is already present which was confirmed by the listing<\/p>\n\n\n\n

root@master>helm repo list\nNAME                    URL                                                                                         \nstable                  https:\/\/charts.helm.sh\/stable                                                               \nelastic                 https:\/\/helm.elastic.co                                                                     \nbitnami                 https:\/\/charts.bitnami.com\/bitnami  <\/code><\/pre>\n\n\n\n
In your case you will have to add<\/code> the repo.<\/p>\n\n\n\n
helm repo add bitnami https:\/\/charts.bitnami.com\/bitnami<\/code><\/pre>\n\n\n\n
Step 2: Deploy the chart<\/h2>\n\n\n\n
helm install my-release bitnami\/kafka<\/code><\/pre>\n\n\n\n
\"\"
Installation<\/figcaption><\/figure>\n\n\n\n
The output identifies the steps you can use to validate the deployment, you can customize as per your needs for the quick test.<\/p>\n\n\n\n
Step 3: Console Producer<\/h2>\n\n\n\n
Before you test you will need to have a client that can produce the events for a specific topic<\/a>.<\/p>\n\n\n\n
kubectl run my-release-kafka-client --restart='Never' --image docker.io\/bitnami\/kafka:3.1.0-debian-10-r14 --namespace default --command -- sleep infinity<\/code><\/pre>\n\n\n\n
You can check the running pod by issuing kubectl get pods<\/code><\/p>\n\n\n\n
my-release-kafka-client                               1\/1     Running   0             22m<\/code><\/pre>\n\n\n\n
Get into to the pod<\/p>\n\n\n\n
 kubectl exec --tty -i my-release-kafka-client --namespace default -- bash<\/code><\/pre>\n\n\n\n
\"\"
Producer<\/figcaption><\/figure>\n\n\n\n
You can either use the same command as outputted by the Helm Install message for validation <\/p>\n\n\n\n
kafka-console-producer.sh \\\n            --broker-list my-release-kafka-0.my-release-kafka-headless.default.svc.cluster.local:9092 \\\n            --topic test<\/code><\/pre>\n\n\n\n
Or customize as per your wish<\/p>\n\n\n\n
> kafka-console-producer.sh --broker-list my-release-kafka-0.my-release-kafka-headless.default.svc.cluster.local:9092 --topic mytopic<\/code><\/pre>\n\n\n\n
Step 4: Console Consumer<\/h2>\n\n\n\n
The information you feed in the producer can be consumed by the client consumer. For that, log into the client<\/p>\n\n\n\n
 kubectl exec --tty -i my-release-kafka-client --namespace default -- bash<\/code><\/pre>\n\n\n\n
And listen to the topic<\/p>\n\n\n\n
kafka-console-consumer.sh \\\n            --bootstrap-server my-release-kafka.default.svc.cluster.local:9092 \\\n            --topic test \\\n            --from-beginning<\/code><\/pre>\n\n\n\n
Or for mytopic<\/code><\/p>\n\n\n\n
kafka-console-consumer.sh --bootstrap-server my-release-kafka.default.svc.cluster.local:9092 --topic mytopic   <\/code><\/pre>\n\n\n\n
\"\"
Consumer<\/figcaption><\/figure>\n\n\n\n
Learn more about use cases here<\/a>.<\/p>official documentation<\/cite><\/blockquote><\/figure>\n","protected":false},"excerpt":{"rendered":"
While I continue to learn more about architecture and design of kafka, a quickstart to deploy & test kafka is a must. Step 1: Add Chart Repo In my local system it is already present which was confirmed by the listing In your case you will have to add the repo. Step 2: Deploy the […]<\/p>\n","protected":false},"author":2,"featured_media":2094,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"image","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[195,34],"tags":[238,106,225],"class_list":["post-2093","post","type-post","status-publish","format-image","has-post-thumbnail","hentry","category-kubernetes","category-technical","tag-deploy","tag-helm","tag-kafka","post_format-post-format-image"],"_links":{"self":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2093","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/comments?post=2093"}],"version-history":[{"count":0,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2093\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media\/2094"}],"wp:attachment":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media?parent=2093"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/categories?post=2093"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/tags?post=2093"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}