Scripts for the Confluent Open Source Platform using Oracle JDK

Running using Docker-Compose

Versions used:

docker --version && docker-compose --version && docker-machine --version

Docker version 17.09.0-ce, build afdb6d4
docker-compose version 1.16.1, build 6d1ac21
docker-machine version 0.12.2, build 9371605

Prerequisites:

If you’re running on Windows or Mac OS X, you’ll need to use Docker Machine to start the Docker host. Docker runs natively on Linux, so the Docker host will be your local machine if you go that route. Otherwise, be sure to allocate at least 6 GBs RAM to the Docker Machine.

• Create the Docker Machine, using 6 GBs and 2 CPUs - docker-machine-create.sh
• Start the Docker Machine - docker-machine-start.sh
• Status of the Docker Machine - docker-machine-status.sh
• Stop the Docker Machine - docker-machine-stop.sh
• Remove the Docker Machine - docker-machine-rm.sh

Don't forget to check the Important Notes/Caveats from the Official Documentation.

1. ZooKeeper:

1.1. Single Apache ZooKeeper node - docker-compose.yml

1.2. Cluster of 3 Apache ZooKeeper nodes - docker-compose.yml

• Check if leader election succeded - cluster-zkX3-check.sh

2. Kafka:

2.1. A single Apache ZooKeeper node, plus a single Apache Kafka node - docker-compose.yml

2.2. Cluster of 3 Apache ZooKeeper nodes, plus 5 Apache Kafka nodes - docker-compose.yml

• Check if ZooKeeper leader election succeded and if all Kafka brokers know about each other - cluster-zkX3-kfkX3-check.sh

3. Kafka, Schema Registry and REST Proxy:

3.1. A single Apache ZooKeeper node, plus a single Apache Kafka node, a Schema Registry node and a REST Proxy node - docker-compose.yml

3.2. Cluster of 3 Apache ZooKeeper nodes, plus 5 Apache Kafka nodes, a Schema Registry node and a REST Proxy node - docker-compose.yml

• Check if ZooKeeper leader election succeded and if all Kafka brokers know about each other - cluster-zkX3-kfkX3-sr-rp-check.sh

Running using Kubernetes:

Versions used:

kubectl version

Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-09T07:27:47Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.8", GitCommit:"bc6162cc70b4a39a7f39391564e0dd0be60b39e9", GitTreeState:"clean", BuildDate:"2017-10-05T06:35:40Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

0. Prerequisites:

Create a dedicated namespace - Namespace

1. ZooKeeper:

• Service
• ConfigMap
• PodDisruptionBudget
• Deployment or StatefulSet

2. Kafka:

• Service
• ConfigMap
• PodDisruptionBudget
• Deployment or StatefulSet

3. Schema Registry:

• Service
• Ingress
• ConfigMap
• PodDisruptionBudget
• Deployment

4. REST Proxy:

• Service
• Ingress
• ConfigMap
• PodDisruptionBudget
• Deployment

Running using Minikube:

Versions used:

minikube version && kubectl version

minikube version: v0.23.0
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-09T07:27:47Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"0b9efaeb34a2fc51ff8e4d34ad9bc6375459c4a4", GitTreeState:"dirty", BuildDate:"2017-10-17T15:09:55Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Prerequisites:

One has to start the minikube using the xhyve driver.

• Start Minikube minikube-start.sh
• Stop Minikube minikube-stop.sh
• Delete Minikube minikube-delete.sh
• Check Minikube dashboard minikube-dashboard.sh
• Check kubectl's current context minikube-check.sh

1. ZooKeeper:

• Service #1
• Service #2
• Service #3
• Pod #1
• Pod #2
• Pod #3

2. Kafka:

• Service #1
• Service #2
• Service #3
• Service #4
• Service #5
• Pod #1
• Pod #2
• Pod #3
• Pod #4
• Pod #5

3. Schema Registry:

• Service #1
• Pod #1

4. REST Proxy:

• Service #1
• Pod #1

Dual licensed under:

• MIT License
• Oracle Binary Code License Agreement