org.apache.hadoop.hbase.zookeeper

Class ZKUtil

java.lang.Object

org.apache.hadoop.hbase.zookeeper.ZKUtil

@InterfaceAudience.Private
public final class ZKUtil
extends Object

Internal HBase utility class for ZooKeeper.

Contains only static methods and constants.

Methods all throw KeeperException if there is an unexpected zookeeper exception, so callers of these methods must handle appropriately. If ZK is required for the operation, the server will need to be aborted.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	ZKUtil.JaasConfiguration A JAAS configuration that defines the login modules that we want to use for login.
static class	ZKUtil.NodeAndData Deprecated. Unused
static class	ZKUtil.ZKUtilOp Represents an action taken by ZKUtil, e.g.

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOG
private static int	zkDumpConnectionTimeOut

Constructor Summary

Constructors

Modifier	Constructor and Description
private	ZKUtil()

Method Summary

Modifier and Type	Method and Description
private static void	appendHFileRefsZNodes(ZKWatcher zkw, String hFileRefsZNode, StringBuilder sb)
private static void	appendPeerState(ZKWatcher zkw, String znodeToProcess, StringBuilder sb)
private static void	appendPeersZnodes(ZKWatcher zkw, String peersZnode, StringBuilder sb)
private static void	appendRSZnodes(ZKWatcher zkw, String znode, StringBuilder sb)
static void	asyncCreate(ZKWatcher zkw, String znode, byte[] data, org.apache.zookeeper.AsyncCallback.StringCallback cb, Object ctx) Async creates the specified node with the specified data.
static int	checkExists(ZKWatcher zkw, String znode) Check if the specified node exists.
static RecoverableZooKeeper	connect(org.apache.hadoop.conf.Configuration conf, String ensemble, org.apache.zookeeper.Watcher watcher)
static RecoverableZooKeeper	connect(org.apache.hadoop.conf.Configuration conf, String ensemble, org.apache.zookeeper.Watcher watcher, String identifier)
static RecoverableZooKeeper	connect(org.apache.hadoop.conf.Configuration conf, org.apache.zookeeper.Watcher watcher) Creates a new connection to ZooKeeper, pulling settings and ensemble config from the specified configuration object using methods from ZKConfig.
static org.apache.zookeeper.KeeperException	convert(DeserializationException e) Convert a DeserializationException to a more palatable KeeperException.
private static ArrayList<org.apache.zookeeper.data.ACL>	createACL(ZKWatcher zkw, String node)
static ArrayList<org.apache.zookeeper.data.ACL>	createACL(ZKWatcher zkw, String node, boolean isSecureZooKeeper)
static void	createAndFailSilent(ZKWatcher zkw, String znode) Creates the specified node, iff the node does not exist.
static void	createAndFailSilent(ZKWatcher zkw, String znode, byte[] data) Creates the specified node containing specified data, iff the node does not exist.
private static void	createAndFailSilent(ZKWatcher zkw, ZKUtil.ZKUtilOp.CreateAndFailSilent cafs)
static int	createAndWatch(ZKWatcher zkw, String znode, byte[] data) Creates the specified node with the specified data and watches it.
static boolean	createEphemeralNodeAndWatch(ZKWatcher zkw, String znode, byte[] data) Set the specified znode to be an ephemeral node carrying the specified data.
static boolean	createNodeIfNotExistsAndWatch(ZKWatcher zkw, String znode, byte[] data) Creates the specified znode to be a persistent node carrying the specified data.
static String	createNodeIfNotExistsNoWatch(ZKWatcher zkw, String znode, byte[] data, org.apache.zookeeper.CreateMode createMode) Creates the specified znode with the specified data but does not watch it.
static void	createSetData(ZKWatcher zkw, String znode, byte[] data) Set data into node creating node if it doesn't yet exist.
static void	createWithParents(ZKWatcher zkw, String znode) Creates the specified node and all parent nodes required for it to exist.
static void	createWithParents(ZKWatcher zkw, String znode, byte[] data) Creates the specified node and all parent nodes required for it to exist.
static void	deleteChildrenRecursively(ZKWatcher zkw, String node) Delete all the children of the specified node but not the node itself.
static void	deleteChildrenRecursivelyMultiOrSequential(ZKWatcher zkw, boolean runSequentialOnMultiFailure, String... pathRoots) Delete all the children of the specified node but not the node itself.
static void	deleteNode(ZKWatcher zkw, String node) Delete the specified node.
static boolean	deleteNode(ZKWatcher zkw, String node, int version) Delete the specified node with the specified version.
static void	deleteNodeFailSilent(ZKWatcher zkw, String node) Deletes the specified node.
private static void	deleteNodeFailSilent(ZKWatcher zkw, ZKUtil.ZKUtilOp.DeleteNodeFailSilent dnfs)
static void	deleteNodeRecursively(ZKWatcher zkw, String node) Delete the specified node and all of it's children.
static void	deleteNodeRecursivelyMultiOrSequential(ZKWatcher zkw, boolean runSequentialOnMultiFailure, String... pathRoots) Delete the specified node and its children.
static String	dump(ZKWatcher zkw)
(package private) static int	estimateSize(ZKUtil.ZKUtilOp op)
static List<ZKUtil.NodeAndData>	getChildDataAndWatchForNewChildren(ZKWatcher zkw, String baseNode) Deprecated. Unused
static List<ZKUtil.NodeAndData>	getChildDataAndWatchForNewChildren(ZKWatcher zkw, String baseNode, boolean throwOnInterrupt) Deprecated. Unused
static byte[]	getData(ZKWatcher zkw, String znode) Get znode data.
static byte[]	getDataAndWatch(ZKWatcher zkw, String znode) Get the data at the specified znode and set a watch.
static byte[]	getDataAndWatch(ZKWatcher zkw, String znode, boolean throwOnInterrupt) Get the data at the specified znode and set a watch.
static byte[]	getDataAndWatch(ZKWatcher zkw, String znode, org.apache.zookeeper.data.Stat stat) Get the data at the specified znode and set a watch.
private static byte[]	getDataInternal(ZKWatcher zkw, String znode, org.apache.zookeeper.data.Stat stat, boolean watcherSet, boolean throwOnInterrupt)
static byte[]	getDataNoWatch(ZKWatcher zkw, String znode, org.apache.zookeeper.data.Stat stat) Get the data at the specified znode without setting a watch.
static String	getNodeName(String path) Get the name of the current node from the specified fully-qualified path.
static int	getNumberOfChildren(ZKWatcher zkw, String znode) Get the number of children of the specified node.
static String	getParent(String node) Returns the full path of the immediate parent of the specified node.
static String	getReplicationZnodesDump(ZKWatcher zkw) Returns a string with replication znodes and position of the replication log
private static void	getReplicationZnodesDump(ZKWatcher zkw, StringBuilder sb) Appends replication znodes to the passed StringBuilder.
private static String	getServerNameOrEmptyString(byte[] data)
private static String[]	getServerStats(String server, int timeout) Gets the statistics from the given server.
static boolean	isSecureZooKeeper(org.apache.hadoop.conf.Configuration conf) Returns whether or not secure authentication is enabled (whether hbase.security.authentication is set to kerberos.
static List<String>	listChildrenAndWatchForNewChildren(ZKWatcher zkw, String znode) Lists the children znodes of the specified znode.
static List<String>	listChildrenAndWatchThem(ZKWatcher zkw, String znode) List all the children of the specified znode, setting a watch for children changes and also setting a watch on every individual child in order to get the NodeCreated and NodeDeleted events.
private static List<String>	listChildrenBFSAndWatchThem(ZKWatcher zkw, String znode) BFS Traversal of all the children under path, with the entries in the list, in the same order as that of the traversal.
private static List<String>	listChildrenBFSNoWatch(ZKWatcher zkw, String znode) BFS Traversal of all the children under path, with the entries in the list, in the same order as that of the traversal.
static List<String>	listChildrenNoWatch(ZKWatcher zkw, String znode) Lists the children of the specified znode without setting any watches.
private static void	login(org.apache.hadoop.conf.Configuration conf, String keytabFileKey, String userNameKey, String hostname, String loginContextProperty, String loginContextName) Log in the current process using the given configuration keys for the credential file and login principal.
static void	loginClient(org.apache.hadoop.conf.Configuration conf, String keytabFileKey, String userNameKey, String hostname) Log in the current zookeeper client using the given configuration keys for the credential file and login principal.
static void	loginServer(org.apache.hadoop.conf.Configuration conf, String keytabFileKey, String userNameKey, String hostname) Log in the current zookeeper server process using the given configuration keys for the credential file and login principal.
private static void	logRetrievedMsg(ZKWatcher zkw, String znode, byte[] data, boolean watcherSet)
static void	logZKTree(ZKWatcher zkw, String root) Recursively print the current state of ZK (non-transactional)
private static void	logZKTree(ZKWatcher zkw, String root, String prefix) Helper method to print the current state of the ZK tree.
static void	multiOrSequential(ZKWatcher zkw, List<ZKUtil.ZKUtilOp> ops, boolean runSequentialOnMultiFailure) Use ZooKeeper's multi-update functionality.
static boolean	nodeHasChildren(ZKWatcher zkw, String znode) Checks if the specified znode has any children.
static long	parseWALPositionFrom(byte[] bytes)
(package private) static List<List<ZKUtil.ZKUtilOp>>	partitionOps(List<ZKUtil.ZKUtilOp> ops, int maxPartitionSize) Partition the list of ops by size (using estimateSize(ZKUtilOp)).
static byte[]	positionToByteArray(long position)
private static void	processSequentially(ZKWatcher zkw, List<ZKUtil.ZKUtilOp> ops)
static void	setData(ZKWatcher zkw, String znode, byte[] data) Sets the data of the existing znode to be the specified data.
static boolean	setData(ZKWatcher zkw, String znode, byte[] data, int expectedVersion) Sets the data of the existing znode to be the specified data.
private static void	setData(ZKWatcher zkw, ZKUtil.ZKUtilOp.SetData setData)
static boolean	setWatchIfNodeExists(ZKWatcher zkw, String znode) Watch the specified znode, but only if exists.
private static void	submitBatchedMultiOrSequential(ZKWatcher zkw, boolean runSequentialOnMultiFailure, List<ZKUtil.ZKUtilOp> ops) Chunks the provided ops when their approximate size exceeds the the configured limit.
private static org.apache.zookeeper.Op	toZooKeeperOp(ZKWatcher zkw, ZKUtil.ZKUtilOp op) Convert from ZKUtilOp to ZKOp
static void	updateExistingNodeData(ZKWatcher zkw, String znode, byte[] data, int expectedVersion) Deprecated. Unused
static void	waitForBaseZNode(org.apache.hadoop.conf.Configuration conf) Waits for HBase installation's base (parent) znode to become available.
static boolean	watchAndCheckExists(ZKWatcher zkw, String znode) Watch the specified znode for delete/create/change events.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

zkDumpConnectionTimeOut

private static int zkDumpConnectionTimeOut

Constructor Detail

ZKUtil

private ZKUtil()

Method Detail

connect

public static RecoverableZooKeeper connect(org.apache.hadoop.conf.Configuration conf,
                                           org.apache.zookeeper.Watcher watcher)
                                    throws IOException

Creates a new connection to ZooKeeper, pulling settings and ensemble config from the specified configuration object using methods from ZKConfig. Sets the connection status monitoring watcher to the specified watcher.

Parameters:

conf - configuration to pull ensemble and other settings from

watcher - watcher to monitor connection changes

Returns:

connection to zookeeper

Throws:

IOException - if unable to connect to zk or config problem

connect

public static RecoverableZooKeeper connect(org.apache.hadoop.conf.Configuration conf,
                                           String ensemble,
                                           org.apache.zookeeper.Watcher watcher)
                                    throws IOException

Throws:

IOException

connect

public static RecoverableZooKeeper connect(org.apache.hadoop.conf.Configuration conf,
                                           String ensemble,
                                           org.apache.zookeeper.Watcher watcher,
                                           String identifier)
                                    throws IOException

Throws:

IOException

loginServer

public static void loginServer(org.apache.hadoop.conf.Configuration conf,
                               String keytabFileKey,
                               String userNameKey,
                               String hostname)
                        throws IOException

Log in the current zookeeper server process using the given configuration keys for the credential file and login principal.

This is only applicable when running on secure hbase On regular HBase (without security features), this will safely be ignored.

Parameters:

conf - The configuration data to use

keytabFileKey - Property key used to configure the path to the credential file

userNameKey - Property key used to configure the login principal

hostname - Current hostname to use in any credentials

Throws:

IOException - underlying exception from SecurityUtil.login() call

loginClient

public static void loginClient(org.apache.hadoop.conf.Configuration conf,
                               String keytabFileKey,
                               String userNameKey,
                               String hostname)
                        throws IOException

Log in the current zookeeper client using the given configuration keys for the credential file and login principal.

This is only applicable when running on secure hbase On regular HBase (without security features), this will safely be ignored.

Parameters:

conf - The configuration data to use

keytabFileKey - Property key used to configure the path to the credential file

userNameKey - Property key used to configure the login principal

hostname - Current hostname to use in any credentials

Throws:

IOException - underlying exception from SecurityUtil.login() call

login

private static void login(org.apache.hadoop.conf.Configuration conf,
                          String keytabFileKey,
                          String userNameKey,
                          String hostname,
                          String loginContextProperty,
                          String loginContextName)
                   throws IOException

Log in the current process using the given configuration keys for the credential file and login principal.

This is only applicable when running on secure hbase On regular HBase (without security features), this will safely be ignored.

Parameters:

conf - The configuration data to use

keytabFileKey - Property key used to configure the path to the credential file

userNameKey - Property key used to configure the login principal

hostname - Current hostname to use in any credentials

loginContextProperty - property name to expose the entry name

loginContextName - jaas entry name

Throws:

IOException - underlying exception from SecurityUtil.login() call

getParent

public static String getParent(String node)

Returns the full path of the immediate parent of the specified node.

Parameters:

node - path to get parent of

Returns:

parent of path, null if passed the root node or an invalid node

getNodeName

public static String getNodeName(String path)

Get the name of the current node from the specified fully-qualified path.

Parameters:

path - fully-qualified path

Returns:

name of the current node

watchAndCheckExists

public static boolean watchAndCheckExists(ZKWatcher zkw,
                                          String znode)
                                   throws org.apache.zookeeper.KeeperException

Watch the specified znode for delete/create/change events. The watcher is set whether or not the node exists. If the node already exists, the method returns true. If the node does not exist, the method returns false.

Parameters:

zkw - zk reference

znode - path of node to watch

Returns:

true if znode exists, false if does not exist or error

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

setWatchIfNodeExists

public static boolean setWatchIfNodeExists(ZKWatcher zkw,
                                           String znode)
                                    throws org.apache.zookeeper.KeeperException

Watch the specified znode, but only if exists. Useful when watching for deletions. Uses .getData() (and handles NoNodeException) instead of .exists() to accomplish this, as .getData() will only set a watch if the znode exists.

Parameters:

zkw - zk reference

znode - path of node to watch

Returns:

true if the watch is set, false if node does not exists

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

checkExists

public static int checkExists(ZKWatcher zkw,
                              String znode)
                       throws org.apache.zookeeper.KeeperException

Check if the specified node exists. Sets no watches.

Parameters:

zkw - zk reference

znode - path of node to watch

Returns:

version of the node if it exists, -1 if does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

listChildrenAndWatchForNewChildren

public static List<String> listChildrenAndWatchForNewChildren(ZKWatcher zkw,
                                                              String znode)
                                                       throws org.apache.zookeeper.KeeperException

Lists the children znodes of the specified znode. Also sets a watch on the specified znode which will capture a NodeDeleted event on the specified znode as well as NodeChildrenChanged if any children of the specified znode are created or deleted. Returns null if the specified node does not exist. Otherwise returns a list of children of the specified node. If the node exists but it has no children, an empty list will be returned.

Parameters:

zkw - zk reference

znode - path of node to list and watch children of

Returns:

list of children of the specified node, an empty list if the node exists but has no children, and null if the node does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

listChildrenAndWatchThem

public static List<String> listChildrenAndWatchThem(ZKWatcher zkw,
                                                    String znode)
                                             throws org.apache.zookeeper.KeeperException

List all the children of the specified znode, setting a watch for children changes and also setting a watch on every individual child in order to get the NodeCreated and NodeDeleted events.

Parameters:

zkw - zookeeper reference

znode - node to get children of and watch

Returns:

list of znode names, null if the node doesn't exist

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

listChildrenNoWatch

public static List<String> listChildrenNoWatch(ZKWatcher zkw,
                                               String znode)
                                        throws org.apache.zookeeper.KeeperException

Lists the children of the specified znode without setting any watches. Sets no watches at all, this method is best effort. Returns an empty list if the node has no children. Returns null if the parent node itself does not exist.

Parameters:

zkw - zookeeper reference

znode - node to get children

Returns:

list of data of children of specified znode, empty if no children, null if parent does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

nodeHasChildren

public static boolean nodeHasChildren(ZKWatcher zkw,
                                      String znode)
                               throws org.apache.zookeeper.KeeperException

Checks if the specified znode has any children. Sets no watches. Returns true if the node exists and has children. Returns false if the node does not exist or if the node does not have any children. Used during master initialization to determine if the master is a failed-over-to master or the first master during initial cluster startup. If the directory for regionserver ephemeral nodes is empty then this is a cluster startup, if not then it is not cluster startup.

Parameters:

zkw - zk reference

znode - path of node to check for children of

Returns:

true if node has children, false if not or node does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

getNumberOfChildren

public static int getNumberOfChildren(ZKWatcher zkw,
                                      String znode)
                               throws org.apache.zookeeper.KeeperException

Get the number of children of the specified node. If the node does not exist or has no children, returns 0. Sets no watches at all.

Parameters:

zkw - zk reference

znode - path of node to count children of

Returns:

number of children of specified node, 0 if none or parent does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

getData

public static byte[] getData(ZKWatcher zkw,
                             String znode)
                      throws org.apache.zookeeper.KeeperException,
                             InterruptedException

Get znode data. Does not set a watcher.

Returns:

ZNode data, null if the node does not exist or if there is an error.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

getDataAndWatch

public static byte[] getDataAndWatch(ZKWatcher zkw,
                                     String znode)
                              throws org.apache.zookeeper.KeeperException

Get the data at the specified znode and set a watch. Returns the data and sets a watch if the node exists. Returns null and no watch is set if the node does not exist or there is an exception.

Parameters:

zkw - zk reference

znode - path of node

Returns:

data of the specified znode, or null

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

getDataAndWatch

public static byte[] getDataAndWatch(ZKWatcher zkw,
                                     String znode,
                                     boolean throwOnInterrupt)
                              throws org.apache.zookeeper.KeeperException

Get the data at the specified znode and set a watch. Returns the data and sets a watch if the node exists. Returns null and no watch is set if the node does not exist or there is an exception.

Parameters:

zkw - zk reference

znode - path of node

throwOnInterrupt - if false then just interrupt the thread, do not throw exception

Returns:

data of the specified znode, or null

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

getDataAndWatch

public static byte[] getDataAndWatch(ZKWatcher zkw,
                                     String znode,
                                     org.apache.zookeeper.data.Stat stat)
                              throws org.apache.zookeeper.KeeperException

Get the data at the specified znode and set a watch. Returns the data and sets a watch if the node exists. Returns null and no watch is set if the node does not exist or there is an exception.

Parameters:

zkw - zk reference

znode - path of node

stat - object to populate the version of the znode

Returns:

data of the specified znode, or null

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

getDataInternal

private static byte[] getDataInternal(ZKWatcher zkw,
                                      String znode,
                                      org.apache.zookeeper.data.Stat stat,
                                      boolean watcherSet,
                                      boolean throwOnInterrupt)
                               throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

getDataNoWatch

public static byte[] getDataNoWatch(ZKWatcher zkw,
                                    String znode,
                                    org.apache.zookeeper.data.Stat stat)
                             throws org.apache.zookeeper.KeeperException

Get the data at the specified znode without setting a watch. Returns the data if the node exists. Returns null if the node does not exist. Sets the stats of the node in the passed Stat object. Pass a null stat if not interested.

Parameters:

zkw - zk reference

znode - path of node

stat - node status to get if node exists

Returns:

data of the specified znode, or null if node does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

getChildDataAndWatchForNewChildren

@Deprecated
public static List<ZKUtil.NodeAndData> getChildDataAndWatchForNewChildren(ZKWatcher zkw,
                                                                                      String baseNode)
                                                                               throws org.apache.zookeeper.KeeperException

Deprecated. Unused

Returns the date of child znodes of the specified znode. Also sets a watch on the specified znode which will capture a NodeDeleted event on the specified znode as well as NodeChildrenChanged if any children of the specified znode are created or deleted. Returns null if the specified node does not exist. Otherwise returns a list of children of the specified node. If the node exists but it has no children, an empty list will be returned.

Parameters:

zkw - zk reference

baseNode - path of node to list and watch children of

Returns:

list of data of children of the specified node, an empty list if the node exists but has no children, and null if the node does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

getChildDataAndWatchForNewChildren

@Deprecated
public static List<ZKUtil.NodeAndData> getChildDataAndWatchForNewChildren(ZKWatcher zkw,
                                                                                      String baseNode,
                                                                                      boolean throwOnInterrupt)
                                                                               throws org.apache.zookeeper.KeeperException

Deprecated. Unused

Returns the date of child znodes of the specified znode. Also sets a watch on the specified znode which will capture a NodeDeleted event on the specified znode as well as NodeChildrenChanged if any children of the specified znode are created or deleted. Returns null if the specified node does not exist. Otherwise returns a list of children of the specified node. If the node exists but it has no children, an empty list will be returned.

Parameters:

zkw - zk reference

baseNode - path of node to list and watch children of

throwOnInterrupt - if true then just interrupt the thread, do not throw exception

Returns:

list of data of children of the specified node, an empty list if the node exists but has no children, and null if the node does not exist

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

updateExistingNodeData

@Deprecated
public static void updateExistingNodeData(ZKWatcher zkw,
                                                      String znode,
                                                      byte[] data,
                                                      int expectedVersion)
                                               throws org.apache.zookeeper.KeeperException

Deprecated. Unused

Update the data of an existing node with the expected version to have the specified data. Throws an exception if there is a version mismatch or some other problem. Sets no watches under any conditions.

Parameters:

zkw - zk reference

znode - the path to the ZNode

data - the data to store in ZooKeeper

expectedVersion - the expected version

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

KeeperException.BadVersionException - if version mismatch

setData

public static boolean setData(ZKWatcher zkw,
                              String znode,
                              byte[] data,
                              int expectedVersion)
                       throws org.apache.zookeeper.KeeperException,
                              org.apache.zookeeper.KeeperException.NoNodeException

Sets the data of the existing znode to be the specified data. Ensures that the current data has the specified expected version.

If the node does not exist, a KeeperException.NoNodeException will be thrown.

If their is a version mismatch, method returns null.

No watches are set but setting data will trigger other watchers of this node.

If there is another problem, a KeeperException will be thrown.

Parameters:

zkw - zk reference

znode - path of node

data - data to set for node

expectedVersion - version expected when setting data

Returns:

true if data set, false if version mismatch

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

org.apache.zookeeper.KeeperException.NoNodeException

createSetData

public static void createSetData(ZKWatcher zkw,
                                 String znode,
                                 byte[] data)
                          throws org.apache.zookeeper.KeeperException

Set data into node creating node if it doesn't yet exist. Does not set watch.

Parameters:

zkw - zk reference

znode - path of node

data - data to set for node

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

setData

public static void setData(ZKWatcher zkw,
                           String znode,
                           byte[] data)
                    throws org.apache.zookeeper.KeeperException,
                           org.apache.zookeeper.KeeperException.NoNodeException

Sets the data of the existing znode to be the specified data. The node must exist but no checks are done on the existing data or version.

If the node does not exist, a KeeperException.NoNodeException will be thrown.

No watches are set but setting data will trigger other watchers of this node.

If there is another problem, a KeeperException will be thrown.

Parameters:

zkw - zk reference

znode - path of node

data - data to set for node

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

org.apache.zookeeper.KeeperException.NoNodeException

setData

private static void setData(ZKWatcher zkw,
                            ZKUtil.ZKUtilOp.SetData setData)
                     throws org.apache.zookeeper.KeeperException,
                            org.apache.zookeeper.KeeperException.NoNodeException

Throws:

org.apache.zookeeper.KeeperException

org.apache.zookeeper.KeeperException.NoNodeException

isSecureZooKeeper

public static boolean isSecureZooKeeper(org.apache.hadoop.conf.Configuration conf)

Returns whether or not secure authentication is enabled (whether hbase.security.authentication is set to kerberos.

createACL

private static ArrayList<org.apache.zookeeper.data.ACL> createACL(ZKWatcher zkw,
                                                                  String node)

createACL

public static ArrayList<org.apache.zookeeper.data.ACL> createACL(ZKWatcher zkw,
                                                                 String node,
                                                                 boolean isSecureZooKeeper)

createEphemeralNodeAndWatch

public static boolean createEphemeralNodeAndWatch(ZKWatcher zkw,
                                                  String znode,
                                                  byte[] data)
                                           throws org.apache.zookeeper.KeeperException

Set the specified znode to be an ephemeral node carrying the specified data. If the node is created successfully, a watcher is also set on the node. If the node is not created successfully because it already exists, this method will also set a watcher on the node. If there is another problem, a KeeperException will be thrown.

Parameters:

zkw - zk reference

znode - path of node

data - data of node

Returns:

true if node created, false if not, watch set in both cases

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

createNodeIfNotExistsAndWatch

public static boolean createNodeIfNotExistsAndWatch(ZKWatcher zkw,
                                                    String znode,
                                                    byte[] data)
                                             throws org.apache.zookeeper.KeeperException

Creates the specified znode to be a persistent node carrying the specified data. Returns true if the node was successfully created, false if the node already existed. If the node is created successfully, a watcher is also set on the node. If the node is not created successfully because it already exists, this method will also set a watcher on the node but return false. If there is another problem, a KeeperException will be thrown.

Parameters:

zkw - zk reference

znode - path of node

data - data of node

Returns:

true if node created, false if not, watch set in both cases

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

createNodeIfNotExistsNoWatch

public static String createNodeIfNotExistsNoWatch(ZKWatcher zkw,
                                                  String znode,
                                                  byte[] data,
                                                  org.apache.zookeeper.CreateMode createMode)
                                           throws org.apache.zookeeper.KeeperException

Creates the specified znode with the specified data but does not watch it. Returns the znode of the newly created node If there is another problem, a KeeperException will be thrown.

Parameters:

zkw - zk reference

znode - path of node

data - data of node

createMode - specifying whether the node to be created is ephemeral and/or sequential

Returns:

true name of the newly created znode or null

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

createAndWatch

public static int createAndWatch(ZKWatcher zkw,
                                 String znode,
                                 byte[] data)
                          throws org.apache.zookeeper.KeeperException,
                                 org.apache.zookeeper.KeeperException.NodeExistsException

Creates the specified node with the specified data and watches it.

Throws an exception if the node already exists.

The node created is persistent and open access.

Returns the version number of the created node if successful.

Parameters:

zkw - zk reference

znode - path of node to create

data - data of node to create

Returns:

version of node created

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

KeeperException.NodeExistsException - if node already exists

asyncCreate

public static void asyncCreate(ZKWatcher zkw,
                               String znode,
                               byte[] data,
                               org.apache.zookeeper.AsyncCallback.StringCallback cb,
                               Object ctx)

Async creates the specified node with the specified data.

Throws an exception if the node already exists.

The node created is persistent and open access.

Parameters:

zkw - zk reference

znode - path of node to create

data - data of node to create

cb - the callback to use for the creation

ctx - the context to use for the creation

createAndFailSilent

public static void createAndFailSilent(ZKWatcher zkw,
                                       String znode)
                                throws org.apache.zookeeper.KeeperException

Creates the specified node, iff the node does not exist. Does not set a watch and fails silently if the node already exists. The node created is persistent and open access.

Parameters:

zkw - zk reference

znode - path of node

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

createAndFailSilent

public static void createAndFailSilent(ZKWatcher zkw,
                                       String znode,
                                       byte[] data)
                                throws org.apache.zookeeper.KeeperException

Creates the specified node containing specified data, iff the node does not exist. Does not set a watch and fails silently if the node already exists. The node created is persistent and open access.

Parameters:

zkw - zk reference

znode - path of node

data - a byte array data to store in the znode

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

createAndFailSilent

private static void createAndFailSilent(ZKWatcher zkw,
                                        ZKUtil.ZKUtilOp.CreateAndFailSilent cafs)
                                 throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

createWithParents

public static void createWithParents(ZKWatcher zkw,
                                     String znode)
                              throws org.apache.zookeeper.KeeperException

Creates the specified node and all parent nodes required for it to exist. No watches are set and no errors are thrown if the node already exists. The nodes created are persistent and open access.

Parameters:

zkw - zk reference

znode - path of node

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

createWithParents

public static void createWithParents(ZKWatcher zkw,
                                     String znode,
                                     byte[] data)
                              throws org.apache.zookeeper.KeeperException

Creates the specified node and all parent nodes required for it to exist. The creation of parent znodes is not atomic with the leafe znode creation but the data is written atomically when the leaf node is created. No watches are set and no errors are thrown if the node already exists. The nodes created are persistent and open access.

Parameters:

zkw - zk reference

znode - path of node

Throws:

org.apache.zookeeper.KeeperException - if unexpected zookeeper exception

deleteNode

public static void deleteNode(ZKWatcher zkw,
                              String node)
                       throws org.apache.zookeeper.KeeperException

Delete the specified node. Sets no watches. Throws all exceptions.

Throws:

org.apache.zookeeper.KeeperException

deleteNode

public static boolean deleteNode(ZKWatcher zkw,
                                 String node,
                                 int version)
                          throws org.apache.zookeeper.KeeperException

Delete the specified node with the specified version. Sets no watches. Throws all exceptions.

Throws:

org.apache.zookeeper.KeeperException

deleteNodeFailSilent

public static void deleteNodeFailSilent(ZKWatcher zkw,
                                        String node)
                                 throws org.apache.zookeeper.KeeperException

Deletes the specified node. Fails silent if the node does not exist.

Parameters:

zkw - reference to the ZKWatcher which also contains configuration and operation

node - the node to delete

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

deleteNodeFailSilent

private static void deleteNodeFailSilent(ZKWatcher zkw,
                                         ZKUtil.ZKUtilOp.DeleteNodeFailSilent dnfs)
                                  throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

deleteNodeRecursively

public static void deleteNodeRecursively(ZKWatcher zkw,
                                         String node)
                                  throws org.apache.zookeeper.KeeperException

Delete the specified node and all of it's children.

If the node does not exist, just returns.

Sets no watches. Throws all exceptions besides dealing with deletion of children.

Throws:

org.apache.zookeeper.KeeperException

deleteChildrenRecursively

public static void deleteChildrenRecursively(ZKWatcher zkw,
                                             String node)
                                      throws org.apache.zookeeper.KeeperException

Delete all the children of the specified node but not the node itself. Sets no watches. Throws all exceptions besides dealing with deletion of children.

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

deleteChildrenRecursivelyMultiOrSequential

public static void deleteChildrenRecursivelyMultiOrSequential(ZKWatcher zkw,
                                                              boolean runSequentialOnMultiFailure,
                                                              String... pathRoots)
                                                       throws org.apache.zookeeper.KeeperException

Delete all the children of the specified node but not the node itself. This will first traverse the znode tree for listing the children and then delete these znodes using multi-update api or sequential based on the specified configurations.

Sets no watches. Throws all exceptions besides dealing with deletion of children.

If the following is true:

• runSequentialOnMultiFailure is true

on calling multi, we get a ZooKeeper exception that can be handled by a sequential call(*), we retry the operations one-by-one (sequentially).

Parameters:

zkw - - zk reference

runSequentialOnMultiFailure - - if true when we get a ZooKeeper exception that could retry the operations one-by-one (sequentially)

pathRoots - - path of the parent node(s)

Throws:

KeeperException.NotEmptyException - if node has children while deleting

org.apache.zookeeper.KeeperException - if unexpected ZooKeeper exception

IllegalArgumentException - if an invalid path is specified

deleteNodeRecursivelyMultiOrSequential

public static void deleteNodeRecursivelyMultiOrSequential(ZKWatcher zkw,
                                                          boolean runSequentialOnMultiFailure,
                                                          String... pathRoots)
                                                   throws org.apache.zookeeper.KeeperException

Delete the specified node and its children. This traverse the znode tree for listing the children and then delete these znodes including the parent using multi-update api or sequential based on the specified configurations.

Sets no watches. Throws all exceptions besides dealing with deletion of children.

If the following is true:

• runSequentialOnMultiFailure is true

on calling multi, we get a ZooKeeper exception that can be handled by a sequential call(*), we retry the operations one-by-one (sequentially).

Parameters:

zkw - - zk reference

runSequentialOnMultiFailure - - if true when we get a ZooKeeper exception that could retry the operations one-by-one (sequentially)

pathRoots - - path of the parent node(s)

Throws:

KeeperException.NotEmptyException - if node has children while deleting

org.apache.zookeeper.KeeperException - if unexpected ZooKeeper exception

IllegalArgumentException - if an invalid path is specified

submitBatchedMultiOrSequential

private static void submitBatchedMultiOrSequential(ZKWatcher zkw,
                                                   boolean runSequentialOnMultiFailure,
                                                   List<ZKUtil.ZKUtilOp> ops)
                                            throws org.apache.zookeeper.KeeperException

Chunks the provided ops when their approximate size exceeds the the configured limit. Take caution that this can ONLY be used for operations where atomicity is not important, e.g. deletions. It must not be used when atomicity of the operations is critical.

Parameters:

zkw - reference to the ZKWatcher which contains configuration and constants

runSequentialOnMultiFailure - if true when we get a ZooKeeper exception that could retry the operations one-by-one (sequentially)

ops - list of ZKUtilOp ZKUtil.ZKUtilOp to partition while submitting batched multi or sequential

Throws:

org.apache.zookeeper.KeeperException - unexpected ZooKeeper Exception / Zookeeper unreachable

partitionOps

static List<List<ZKUtil.ZKUtilOp>> partitionOps(List<ZKUtil.ZKUtilOp> ops,
                                                int maxPartitionSize)

Partition the list of ops by size (using estimateSize(ZKUtilOp)).

estimateSize

static int estimateSize(ZKUtil.ZKUtilOp op)

listChildrenBFSNoWatch

private static List<String> listChildrenBFSNoWatch(ZKWatcher zkw,
                                                   String znode)
                                            throws org.apache.zookeeper.KeeperException

BFS Traversal of all the children under path, with the entries in the list, in the same order as that of the traversal. Lists all the children without setting any watches.

Parameters:

zkw - - zk reference

znode - - path of node

Returns:

list of children znodes under the path

Throws:

org.apache.zookeeper.KeeperException - if unexpected ZooKeeper exception

listChildrenBFSAndWatchThem

private static List<String> listChildrenBFSAndWatchThem(ZKWatcher zkw,
                                                        String znode)
                                                 throws org.apache.zookeeper.KeeperException

BFS Traversal of all the children under path, with the entries in the list, in the same order as that of the traversal. Lists all the children and set watches on to them.

Parameters:

zkw - - zk reference

znode - - path of node

Returns:

list of children znodes under the path

Throws:

org.apache.zookeeper.KeeperException - if unexpected ZooKeeper exception

toZooKeeperOp

private static org.apache.zookeeper.Op toZooKeeperOp(ZKWatcher zkw,
                                                     ZKUtil.ZKUtilOp op)
                                              throws UnsupportedOperationException

Convert from ZKUtilOp to ZKOp

Throws:

UnsupportedOperationException

multiOrSequential

public static void multiOrSequential(ZKWatcher zkw,
                                     List<ZKUtil.ZKUtilOp> ops,
                                     boolean runSequentialOnMultiFailure)
                              throws org.apache.zookeeper.KeeperException

Use ZooKeeper's multi-update functionality. If all of the following are true: - runSequentialOnMultiFailure is true - on calling multi, we get a ZooKeeper exception that can be handled by a sequential call(*) Then: - we retry the operations one-by-one (sequentially) Note *: an example is receiving a NodeExistsException from a "create" call. Without multi, a user could call "createAndFailSilent" to ensure that a node exists if they don't care who actually created the node (i.e. the NodeExistsException from ZooKeeper is caught). This will cause all operations in the multi to fail, however, because the NodeExistsException that zk.create throws will fail the multi transaction. In this case, if the previous conditions hold, the commands are run sequentially, which should result in the correct final state, but means that the operations will not run atomically.

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

processSequentially

private static void processSequentially(ZKWatcher zkw,
                                        List<ZKUtil.ZKUtilOp> ops)
                                 throws org.apache.zookeeper.KeeperException,
                                        org.apache.zookeeper.KeeperException.NoNodeException

Throws:

org.apache.zookeeper.KeeperException

org.apache.zookeeper.KeeperException.NoNodeException

dump

public static String dump(ZKWatcher zkw)

Returns:

String dump of everything in ZooKeeper.

getReplicationZnodesDump

private static void getReplicationZnodesDump(ZKWatcher zkw,
                                             StringBuilder sb)
                                      throws org.apache.zookeeper.KeeperException

Appends replication znodes to the passed StringBuilder.

Parameters:

zkw - reference to the ZKWatcher which also contains configuration and operation

sb - the StringBuilder to append to

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

appendHFileRefsZNodes

private static void appendHFileRefsZNodes(ZKWatcher zkw,
                                          String hFileRefsZNode,
                                          StringBuilder sb)
                                   throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

getReplicationZnodesDump

public static String getReplicationZnodesDump(ZKWatcher zkw)
                                       throws org.apache.zookeeper.KeeperException

Returns a string with replication znodes and position of the replication log

Parameters:

zkw - reference to the ZKWatcher which also contains configuration and operation

Returns:

aq string of replication znodes and log positions

Throws:

org.apache.zookeeper.KeeperException

appendRSZnodes

private static void appendRSZnodes(ZKWatcher zkw,
                                   String znode,
                                   StringBuilder sb)
                            throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

appendPeersZnodes

private static void appendPeersZnodes(ZKWatcher zkw,
                                      String peersZnode,
                                      StringBuilder sb)
                               throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

appendPeerState

private static void appendPeerState(ZKWatcher zkw,
                                    String znodeToProcess,
                                    StringBuilder sb)
                             throws org.apache.zookeeper.KeeperException,
                                    org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException

Throws:

org.apache.zookeeper.KeeperException

org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException

getServerStats

private static String[] getServerStats(String server,
                                       int timeout)
                                throws IOException

Gets the statistics from the given server.

Parameters:

server - The server to get the statistics from.

timeout - The socket timeout to use.

Returns:

The array of response strings.

Throws:

IOException - When the socket communication fails.

logRetrievedMsg

private static void logRetrievedMsg(ZKWatcher zkw,
                                    String znode,
                                    byte[] data,
                                    boolean watcherSet)

getServerNameOrEmptyString

private static String getServerNameOrEmptyString(byte[] data)

waitForBaseZNode

public static void waitForBaseZNode(org.apache.hadoop.conf.Configuration conf)
                             throws IOException

Waits for HBase installation's base (parent) znode to become available.

Throws:

IOException - on ZK errors

convert

public static org.apache.zookeeper.KeeperException convert(DeserializationException e)

Convert a DeserializationException to a more palatable KeeperException. Used when can't let a DeserializationException out w/o changing public API.

Parameters:

e - Exception to convert

Returns:

Converted exception

logZKTree

public static void logZKTree(ZKWatcher zkw,
                             String root)

Recursively print the current state of ZK (non-transactional)

Parameters:

root - name of the root directory in zk to print

logZKTree

private static void logZKTree(ZKWatcher zkw,
                              String root,
                              String prefix)
                       throws org.apache.zookeeper.KeeperException

Helper method to print the current state of the ZK tree.

Throws:

org.apache.zookeeper.KeeperException - if an unexpected exception occurs

See Also:

logZKTree(ZKWatcher, String)

positionToByteArray

public static byte[] positionToByteArray(long position)

Parameters:

position - the position to serialize

Returns:

Serialized protobuf of position with pb magic prefix prepended suitable for use as content of an wal position in a replication queue.

parseWALPositionFrom

public static long parseWALPositionFrom(byte[] bytes)
                                 throws DeserializationException

Parameters:

bytes - - Content of a WAL position znode.

Returns:

long - The current WAL position.

Throws:

DeserializationException - if the WAL position cannot be parsed