In this article we will create a Apache Zookeeper cluster on three machines, we have three ubuntu machies with 1 GB ram.

X.69.97.1
X.34.3.23
X.34.164.220


Before starting, make sure you have Java7+ running on each instance, check java version with following command:

java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

If not installed already follow these Instructions


In order to setup Zookeeper cluster, following steps need to be followed:

Step 1) Download and unzip zookeepr package in all three machines as shown below:

cd /opt/
wget http://mirror.fibergrid.in/apache/zookeeper/stable/zookeeper-3.4.9.tar.gz
tar -zxbf zookeeper-3.4.9.tar.gz


This will create a untar zookeepr package folder with name zookeeper-3.4.9:

cd
zookeeper-3.4.9 zookeeper-3.4.9.tar.gz


Step 2) Configure zoo.cfg file in all three machines as shown below:

You will see a sample conf file with name zoo_sample.cfg, make a copy of it with name zoo.cfg as shown below, and edit new zoo.cfg as described:

cp zoo_sample.cfg zoo.cfg
vim zoo_sample.cfg


you will see something like this:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1


Now change dataDir to some permanant location, 'opt/data' and add following cluster related configuration to the end of file:

server.1=0.0.0.0 :2888:3888
server.2=X .34.3.23:2888:3888
server.3=X .34.164.220:2888:3888


The two ports listed in each server definition are respectively the quorum and election ports used by ZooKeeper nodes to communicate with one another internally. There's usually no need to modify these ports, and you should try to keep them the same across servers for consistency.

Make sure to add '0.0.0.0' as ip in place of machine own ip in all three nodes.

Here 1,2,3 are unique ids of all three nodes in the cluster, make these changes to all three machines part of cluster.

Now the modified file will look like this:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/opt/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=0.0.0.0:2888:3888
server.2=X.34.3.23:2888:3888
server.3=X .34.164.220:2888:3888


Step 3) Create mentioned data directory in all three nodes as shown below:

cd /opt/
mkdir data


Now create a file named 'myid' in each node's data directory and put numbers 1,2,3 in it respectively this will indicate node id in the cluster.

vim myid


Step 4) Start zookeeper in all three nodes one by one, using following command and passing conf file as parameter

bin/zkServer.sh start conf/zoo.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo.cfg
Starting zookeeper ... STARTED


Now a log file will be created in home directory of zookeeper named 'zookeeper.out ', tail the file to see logs:

tail -f zookeeper.out

That's it, we are done with configuring a zookeeper cluster on three ubuntu instances. More conf file changes and other related information can be found on zookeeper official site.
  • By Techburps.com
  • Oct 16, 2016
  • Big Data