org.apache.hadoop.hbase.zookeeper

Class ZKWatcher

java.lang.Object

org.apache.hadoop.hbase.zookeeper.ZKWatcher

All Implemented Interfaces:

Closeable, AutoCloseable, Abortable, org.apache.zookeeper.Watcher

@InterfaceAudience.Private
public class ZKWatcher
extends Object
implements org.apache.zookeeper.Watcher, Abortable, Closeable

Acts as the single ZooKeeper Watcher. One instance of this is instantiated for each Master, RegionServer, and client process.

This is the only class that implements Watcher. Other internal classes which need to be notified of ZooKeeper events must register with the local instance of this watcher via registerListener(org.apache.hadoop.hbase.zookeeper.ZKListener).

This class also holds and manages the connection to ZooKeeper. Code to deal with connection related events and exceptions are handled here.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.zookeeper.Watcher

org.apache.zookeeper.Watcher.Event, org.apache.zookeeper.Watcher.WatcherType

Field Summary

Fields

Modifier and Type	Field and Description
protected Abortable	abortable
private boolean	aborted
private org.apache.hadoop.conf.Configuration	conf
private String	identifier
private List<ZKListener>	listeners
private static org.slf4j.Logger	LOG
private static Pattern	NAME_PATTERN
private String	prefix
private String	quorum
private RecoverableZooKeeper	recoverableZooKeeper
private ExecutorService	zkEventProcessor
private long	zkSyncTimeout
private ZNodePaths	znodePaths

Constructor Summary

Constructors

Constructor and Description
ZKWatcher(org.apache.hadoop.conf.Configuration conf, String identifier, Abortable abortable) Instantiate a ZooKeeper connection and watcher.
ZKWatcher(org.apache.hadoop.conf.Configuration conf, String identifier, Abortable abortable, boolean canCreateBaseZNode) Instantiate a ZooKeeper connection and watcher.
ZKWatcher(org.apache.hadoop.conf.Configuration conf, String identifier, Abortable abortable, boolean canCreateBaseZNode, boolean clientZK) Instantiate a ZooKeeper connection and watcher.

Method Summary

Modifier and Type	Method and Description
void	abort(String why, Throwable e) Abort the server or client.
private boolean	checkACLForSuperUsers(String[] superUsers, List<org.apache.zookeeper.data.ACL> acls)
void	checkAndSetZNodeAcls() On master start, we check the znode ACLs under the root directory and set the ACLs properly if needed.
void	close() Close the connection to ZooKeeper.
private void	connectionEvent(org.apache.zookeeper.WatchedEvent event) Called when there is a connection-related event via the Watcher callback.
List<org.apache.zookeeper.data.ACL>	createACL(String node)
List<org.apache.zookeeper.data.ACL>	createACL(String node, boolean isSecureZooKeeper)
private void	createBaseZNodes()
private List<String>	filterMetaReplicaNodes(List<String> nodes)
org.apache.hadoop.conf.Configuration	getConfiguration()
List<ZKListener>	getListeners() Get a copy of current registered listeners
List<String>	getMetaReplicaNodes() Get the znodes corresponding to the meta replicas from ZK
List<String>	getMetaReplicaNodesAndWatchChildren() Same as getMetaReplicaNodes() except that this also registers a watcher on base znode for subsequent CREATE/DELETE operations on child nodes.
int	getNumberOfListeners() Returns The number of currently registered listeners
String	getQuorum() Get the quorum address of this instance.
RecoverableZooKeeper	getRecoverableZooKeeper() Get the connection to ZooKeeper.
ZNodePaths	getZNodePaths() Get the znodePaths.
void	interruptedException(InterruptedException ie) Handles InterruptedExceptions in client calls.
void	interruptedExceptionNoThrow(InterruptedException ie, boolean throwLater) Log the InterruptedException and interrupt current thread
boolean	isAborted() Check if the server or client was aborted.
private boolean	isBaseZnodeAclSetup(List<org.apache.zookeeper.data.ACL> acls) Checks whether the ACLs returned from the base znode (/hbase) is set for secure setup.
static boolean	isSuperUserId(String[] superUsers, org.apache.zookeeper.data.Id id)
void	keeperException(org.apache.zookeeper.KeeperException ke) Handles KeeperExceptions in client calls.
String	prefix(String str) Adds this instance's identifier as a prefix to the passed str
void	process(org.apache.zookeeper.WatchedEvent event) Method called from ZooKeeper for events and connection status.
private void	processEvent(org.apache.zookeeper.WatchedEvent event)
void	reconnectAfterExpiration()
void	registerListener(ZKListener listener) Register the specified listener to receive ZooKeeper events.
void	registerListenerFirst(ZKListener listener) Register the specified listener to receive ZooKeeper events and add it as the first in the list of current listeners.
private void	setZnodeAclsRecursive(String znode) Set the znode perms recursively.
void	syncOrTimeout(String path) Forces a synchronization of this ZooKeeper client connection within a timeout.
String	toString()
void	unregisterAllListeners() Clean all existing listeners
void	unregisterListener(ZKListener listener)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.Abortable

abort

Field Detail

LOG

private static final org.slf4j.Logger LOG

prefix

private final String prefix

identifier

private String identifier

quorum

private final String quorum

recoverableZooKeeper

private final RecoverableZooKeeper recoverableZooKeeper

abortable

protected Abortable abortable

aborted

private boolean aborted

znodePaths

private final ZNodePaths znodePaths

listeners

private final List<ZKListener> listeners

zkEventProcessor

private final ExecutorService zkEventProcessor

conf

private final org.apache.hadoop.conf.Configuration conf

zkSyncTimeout

private final long zkSyncTimeout

NAME_PATTERN

private static final Pattern NAME_PATTERN

Constructor Detail

ZKWatcher

public ZKWatcher(org.apache.hadoop.conf.Configuration conf,
                 String identifier,
                 Abortable abortable)
          throws ZooKeeperConnectionException,
                 IOException

Instantiate a ZooKeeper connection and watcher.

Parameters:

identifier - string that is passed to RecoverableZookeeper to be used as identifier for this instance. Use null for default.

Throws:

IOException - if the connection to ZooKeeper fails

ZooKeeperConnectionException - if the client can't connect to ZooKeeper

ZKWatcher

public ZKWatcher(org.apache.hadoop.conf.Configuration conf,
                 String identifier,
                 Abortable abortable,
                 boolean canCreateBaseZNode)
          throws IOException,
                 ZooKeeperConnectionException

Instantiate a ZooKeeper connection and watcher.

Parameters:

conf - the configuration to use

identifier - string that is passed to RecoverableZookeeper to be used as identifier for this instance. Use null for default.

abortable - Can be null if there is on error there is no host to abort: e.g. client context.

canCreateBaseZNode - true if a base ZNode can be created

Throws:

IOException - if the connection to ZooKeeper fails

ZooKeeperConnectionException - if the client can't connect to ZooKeeper

ZKWatcher

public ZKWatcher(org.apache.hadoop.conf.Configuration conf,
                 String identifier,
                 Abortable abortable,
                 boolean canCreateBaseZNode,
                 boolean clientZK)
          throws IOException,
                 ZooKeeperConnectionException

Instantiate a ZooKeeper connection and watcher.

Parameters:

conf - the configuration to use

identifier - string that is passed to RecoverableZookeeper to be used as identifier for this instance. Use null for default.

abortable - Can be null if there is on error there is no host to abort: e.g. client context.

canCreateBaseZNode - true if a base ZNode can be created

clientZK - whether this watcher is set to access client ZK

Throws:

IOException - if the connection to ZooKeeper fails

ZooKeeperConnectionException - if the connection to Zookeeper fails when create base ZNodes

Method Detail

createACL

public List<org.apache.zookeeper.data.ACL> createACL(String node)

createACL

public List<org.apache.zookeeper.data.ACL> createACL(String node,
                                                     boolean isSecureZooKeeper)

createBaseZNodes

private void createBaseZNodes()
                       throws ZooKeeperConnectionException

Throws:

ZooKeeperConnectionException

checkAndSetZNodeAcls

public void checkAndSetZNodeAcls()

On master start, we check the znode ACLs under the root directory and set the ACLs properly if needed. If the cluster goes from an unsecure setup to a secure setup, this step is needed so that the existing znodes created with open permissions are now changed with restrictive perms.

setZnodeAclsRecursive

private void setZnodeAclsRecursive(String znode)
                            throws org.apache.zookeeper.KeeperException,
                                   InterruptedException

Set the znode perms recursively. This will do post-order recursion, so that baseZnode ACLs will be set last in case the master fails in between.

Parameters:

znode - the ZNode to set the permissions for

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

isBaseZnodeAclSetup

private boolean isBaseZnodeAclSetup(List<org.apache.zookeeper.data.ACL> acls)
                             throws IOException

Checks whether the ACLs returned from the base znode (/hbase) is set for secure setup.

Parameters:

acls - acls from zookeeper

Returns:

whether ACLs are set for the base znode

Throws:

IOException - if getting the current user fails

checkACLForSuperUsers

private boolean checkACLForSuperUsers(String[] superUsers,
                                      List<org.apache.zookeeper.data.ACL> acls)

isSuperUserId

public static boolean isSuperUserId(String[] superUsers,
                                    org.apache.zookeeper.data.Id id)

toString

public String toString()

Overrides:

toString in class Object

prefix

public String prefix(String str)

Adds this instance's identifier as a prefix to the passed str

Parameters:

str - String to amend.

Returns:

A new string with this instance's identifier as prefix: e.g. if passed 'hello world', the returned string could be

getMetaReplicaNodes

public List<String> getMetaReplicaNodes()
                                 throws org.apache.zookeeper.KeeperException

Get the znodes corresponding to the meta replicas from ZK

Returns:

list of znodes

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

getMetaReplicaNodesAndWatchChildren

public List<String> getMetaReplicaNodesAndWatchChildren()
                                                 throws org.apache.zookeeper.KeeperException

Same as getMetaReplicaNodes() except that this also registers a watcher on base znode for subsequent CREATE/DELETE operations on child nodes.

Throws:

org.apache.zookeeper.KeeperException

filterMetaReplicaNodes

private List<String> filterMetaReplicaNodes(List<String> nodes)

Parameters:

nodes - Input list of znodes

Returns:

Filtered list of znodes from nodes that belong to meta replica(s).

registerListener

public void registerListener(ZKListener listener)

Register the specified listener to receive ZooKeeper events.

Parameters:

listener - the listener to register

registerListenerFirst

public void registerListenerFirst(ZKListener listener)

Register the specified listener to receive ZooKeeper events and add it as the first in the list of current listeners.

Parameters:

listener - the listener to register

unregisterListener

public void unregisterListener(ZKListener listener)

unregisterAllListeners

public void unregisterAllListeners()

Clean all existing listeners

getListeners

public List<ZKListener> getListeners()

Get a copy of current registered listeners

getNumberOfListeners

public int getNumberOfListeners()

Returns The number of currently registered listeners

getRecoverableZooKeeper

public RecoverableZooKeeper getRecoverableZooKeeper()

Get the connection to ZooKeeper.

Returns:

connection reference to zookeeper

reconnectAfterExpiration

public void reconnectAfterExpiration()
                              throws IOException,
                                     org.apache.zookeeper.KeeperException,
                                     InterruptedException

Throws:

IOException

org.apache.zookeeper.KeeperException

InterruptedException

getQuorum

public String getQuorum()

Get the quorum address of this instance.

Returns:

quorum string of this zookeeper connection instance

getZNodePaths

public ZNodePaths getZNodePaths()

Get the znodePaths.

Mainly used for mocking as mockito can not mock a field access.

processEvent

private void processEvent(org.apache.zookeeper.WatchedEvent event)

process

public void process(org.apache.zookeeper.WatchedEvent event)

Method called from ZooKeeper for events and connection status.

Valid events are passed along to listeners. Connection status changes are dealt with locally.

Specified by:

process in interface org.apache.zookeeper.Watcher

connectionEvent

private void connectionEvent(org.apache.zookeeper.WatchedEvent event)

Called when there is a connection-related event via the Watcher callback.

If Disconnected or Expired, this should shutdown the cluster. But, since we send a KeeperException.SessionExpiredException along with the abort call, it's possible for the Abortable to catch it and try to create a new session with ZooKeeper. This is what the client does in HCM.

Parameters:

event - the connection-related event

syncOrTimeout

public void syncOrTimeout(String path)
                   throws org.apache.zookeeper.KeeperException

Forces a synchronization of this ZooKeeper client connection within a timeout. Enforcing a timeout lets the callers fail-fast rather than wait forever for the sync to finish.

Executing this method before running other methods will ensure that the subsequent operations are up-to-date and consistent as of the time that the sync is complete.

This is used for compareAndSwap type operations where we need to read the data of an existing node and delete or transition that node, utilizing the previously read version and data. We want to ensure that the version read is up-to-date from when we begin the operation.

Throws:

org.apache.zookeeper.KeeperException

keeperException

public void keeperException(org.apache.zookeeper.KeeperException ke)
                     throws org.apache.zookeeper.KeeperException

Handles KeeperExceptions in client calls.

This may be temporary but for now this gives one place to deal with these.

TODO: Currently this method rethrows the exception to let the caller handle

Parameters:

ke - the exception to rethrow

Throws:

org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

interruptedException

public void interruptedException(InterruptedException ie)
                          throws org.apache.zookeeper.KeeperException

Handles InterruptedExceptions in client calls.

Parameters:

ie - the InterruptedException instance thrown

Throws:

org.apache.zookeeper.KeeperException - the exception to throw, transformed from the InterruptedException

interruptedExceptionNoThrow

public void interruptedExceptionNoThrow(InterruptedException ie,
                                        boolean throwLater)

Log the InterruptedException and interrupt current thread

Parameters:

ie - The IterruptedException to log

throwLater - Whether we will throw the exception latter

close

public void close()

Close the connection to ZooKeeper.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

abort

public void abort(String why,
                  Throwable e)

Description copied from interface: Abortable

Abort the server or client.

Specified by:

abort in interface Abortable

Parameters:

why - Why we're aborting.

e - Throwable that caused abort. Can be null.

isAborted

public boolean isAborted()

Description copied from interface: Abortable

Check if the server or client was aborted.

Specified by:

isAborted in interface Abortable

Returns:

true if the server or client was aborted, false otherwise