Package org.apache.hadoop.hbase.zookeeper

Class RecoverableZooKeeper

java.lang.Object
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
@Private public class RecoverableZooKeeper extends Object
A zookeeper that can handle 'recoverable' errors.

To handle recoverable errors, developers need to realize that there are two classes of requests: idempotent and non-idempotent requests. Read requests and unconditional sets and deletes are examples of idempotent requests, they can be reissued with the same results.

(Although, the delete may throw a NoNodeException on reissue its effect on the ZooKeeper state is the same.) Non-idempotent requests need special handling, application and library writers need to keep in mind that they may need to encode information in the data or name of znodes to detect retries. A simple example is a create that uses a sequence flag. If a process issues a create("/x-", ..., SEQUENCE) and gets a connection loss exception, that process will reissue another create("/x-", ..., SEQUENCE) and get back x-111. When the process does a getChildren("/"), it sees x-1,x-30,x-109,x-110,x-111, now it could be that x-109 was the result of the previous create, so the process actually owns both x-109 and x-111. An easy way around this is to use "x-process id-" when doing the create. If the process is using an id of 352, before reissuing the create it will do a getChildren("/") and see "x-222-1", "x-542-30", "x-352-109", x-333-110". The process will know that the original create succeeded an the znode it created is "x-352-109".

See Also:
  • "https://cwiki.apache.org/confluence/display/HADOOP2/ZooKeeper+ErrorHandling"

  • Field Details

  • Constructor Details

    • RecoverableZooKeeper

      RecoverableZooKeeper(String quorumServers, int sessionTimeout, org.apache.zookeeper.Watcher watcher, int maxRetries, int retryIntervalMillis, int maxSleepTime, String identifier, int maxMultiSize, org.apache.zookeeper.client.ZKClientConfig zkClientConfig) throws IOException
      Throws:
      IOException

  • Method Details

    • connect

      public static RecoverableZooKeeper connect(org.apache.hadoop.conf.Configuration conf, org.apache.zookeeper.Watcher watcher) throws IOException
      See connect(Configuration, String, Watcher, String, ZKClientConfig).
      Throws:
      IOException

    • connect

      public static RecoverableZooKeeper connect(org.apache.hadoop.conf.Configuration conf, String ensemble, org.apache.zookeeper.Watcher watcher, String identifier, org.apache.zookeeper.client.ZKClientConfig zkClientConfig) throws IOException
      Creates a new connection to ZooKeeper, pulling settings and ensemble config from the specified configuration object using methods from ZKConfig. Sets the connection status monitoring watcher to the specified watcher.
      Parameters:
      conf - configuration to pull ensemble and other settings from
      watcher - watcher to monitor connection changes
      ensemble - ZooKeeper servers quorum string
      identifier - value used to identify this client instance.
      zkClientConfig - client specific configurations for this instance
      Returns:
      connection to zookeeper
      Throws:
      IOException - if unable to connect to zk or config problem

    • getMaxMultiSizeLimit

      public int getMaxMultiSizeLimit()
      Returns the maximum size (in bytes) that should be included in any single multi() call. NB: This is an approximation, so there may be variance in the msg actually sent over the wire. Please be sure to set this approximately, with respect to your ZK server configuration for jute.maxbuffer.

    • checkZk

      private org.apache.zookeeper.ZooKeeper checkZk() throws org.apache.zookeeper.KeeperException
      Try to create a ZooKeeper connection. Turns any exception encountered into a KeeperException.OperationTimeoutException so it can retried.
      Returns:
      The created ZooKeeper connection object
      Throws:
      org.apache.zookeeper.KeeperException - if a ZooKeeper operation fails

    • reconnectAfterExpiration

      public void reconnectAfterExpiration() throws IOException, org.apache.zookeeper.KeeperException, InterruptedException
      Throws:
      IOException
      org.apache.zookeeper.KeeperException
      InterruptedException

    • delete

      public void delete(String path, int version) throws InterruptedException, org.apache.zookeeper.KeeperException
      delete is an idempotent operation. Retry before throwing exception. This function will not throw NoNodeException if the path does not exist.
      Throws:
      InterruptedException
      org.apache.zookeeper.KeeperException

    • exists

      public org.apache.zookeeper.data.Stat exists(String path, org.apache.zookeeper.Watcher watcher) throws org.apache.zookeeper.KeeperException, InterruptedException
      exists is an idempotent operation. Retry before throwing exception
      Returns:
      A Stat instance
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • exists

      private org.apache.zookeeper.data.Stat exists(String path, org.apache.zookeeper.Watcher watcher, Boolean watch) throws InterruptedException, org.apache.zookeeper.KeeperException
      Throws:
      InterruptedException
      org.apache.zookeeper.KeeperException

    • exists

      public org.apache.zookeeper.data.Stat exists(String path, boolean watch) throws org.apache.zookeeper.KeeperException, InterruptedException
      exists is an idempotent operation. Retry before throwing exception
      Returns:
      A Stat instance
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • retryOrThrow

      private void retryOrThrow(RetryCounter retryCounter, org.apache.zookeeper.KeeperException e, String opName) throws org.apache.zookeeper.KeeperException
      Throws:
      org.apache.zookeeper.KeeperException

    • getChildren

      public List<String> getChildren(String path, org.apache.zookeeper.Watcher watcher) throws org.apache.zookeeper.KeeperException, InterruptedException
      getChildren is an idempotent operation. Retry before throwing exception
      Returns:
      List of children znodes
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • getChildren

      private List<String> getChildren(String path, org.apache.zookeeper.Watcher watcher, Boolean watch) throws InterruptedException, org.apache.zookeeper.KeeperException
      Throws:
      InterruptedException
      org.apache.zookeeper.KeeperException

    • getChildren

      public List<String> getChildren(String path, boolean watch) throws org.apache.zookeeper.KeeperException, InterruptedException
      getChildren is an idempotent operation. Retry before throwing exception
      Returns:
      List of children znodes
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • getData

      public byte[] getData(String path, org.apache.zookeeper.Watcher watcher, org.apache.zookeeper.data.Stat stat) throws org.apache.zookeeper.KeeperException, InterruptedException
      getData is an idempotent operation. Retry before throwing exception
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • getData

      private byte[] getData(String path, org.apache.zookeeper.Watcher watcher, Boolean watch, org.apache.zookeeper.data.Stat stat) throws InterruptedException, org.apache.zookeeper.KeeperException
      Throws:
      InterruptedException
      org.apache.zookeeper.KeeperException

    • getData

      public byte[] getData(String path, boolean watch, org.apache.zookeeper.data.Stat stat) throws org.apache.zookeeper.KeeperException, InterruptedException
      getData is an idempotent operation. Retry before throwing exception
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • setData

      public org.apache.zookeeper.data.Stat setData(String path, byte[] data, int version) throws org.apache.zookeeper.KeeperException, InterruptedException
      setData is NOT an idempotent operation. Retry may cause BadVersion Exception Adding an identifier field into the data to check whether badversion is caused by the result of previous correctly setData
      Returns:
      Stat instance
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • getAcl

      public List<org.apache.zookeeper.data.ACL> getAcl(String path, org.apache.zookeeper.data.Stat stat) throws org.apache.zookeeper.KeeperException, InterruptedException
      getAcl is an idempotent operation. Retry before throwing exception
      Returns:
      list of ACLs
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • setAcl

      public org.apache.zookeeper.data.Stat setAcl(String path, List<org.apache.zookeeper.data.ACL> acls, int version) throws org.apache.zookeeper.KeeperException, InterruptedException
      setAcl is an idempotent operation. Retry before throwing exception
      Returns:
      list of ACLs
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • create

      public String create(String path, byte[] data, List<org.apache.zookeeper.data.ACL> acl, org.apache.zookeeper.CreateMode createMode) throws org.apache.zookeeper.KeeperException, InterruptedException

      NONSEQUENTIAL create is idempotent operation. Retry before throwing exceptions. But this function will not throw the NodeExist exception back to the application.

      But SEQUENTIAL is NOT idempotent operation. It is necessary to add identifier to the path to verify, whether the previous one is successful or not.

      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • createNonSequential

      private String createNonSequential(String path, byte[] data, List<org.apache.zookeeper.data.ACL> acl, org.apache.zookeeper.CreateMode createMode) throws org.apache.zookeeper.KeeperException, InterruptedException
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • createSequential

      private String createSequential(String path, byte[] data, List<org.apache.zookeeper.data.ACL> acl, org.apache.zookeeper.CreateMode createMode) throws org.apache.zookeeper.KeeperException, InterruptedException
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • prepareZKMulti

      private Iterable<org.apache.zookeeper.Op> prepareZKMulti(Iterable<org.apache.zookeeper.Op> ops) throws UnsupportedOperationException
      Convert Iterable of Op we got into the ZooKeeper.Op instances to actually pass to multi (need to do this in order to appendMetaData).
      Throws:
      UnsupportedOperationException

    • multi

      public List<org.apache.zookeeper.OpResult> multi(Iterable<org.apache.zookeeper.Op> ops) throws org.apache.zookeeper.KeeperException, InterruptedException
      Run multiple operations in a transactional manner. Retry before throwing exception
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • findPreviousSequentialNode

      private String findPreviousSequentialNode(String path) throws org.apache.zookeeper.KeeperException, InterruptedException
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • getSessionId

      public long getSessionId()

    • close

      public void close() throws InterruptedException
      Throws:
      InterruptedException

    • getState

      public org.apache.zookeeper.ZooKeeper.States getState()

    • getZooKeeper

      public org.apache.zookeeper.ZooKeeper getZooKeeper()
      Returns the wrapped ZooKeeper client. If the wrapped client hasn't been created yet then tries create it first.
      Returns:
      the wrapped ZK client of null if the creation has failed.

    • sync

      public void sync(String path, org.apache.zookeeper.AsyncCallback.VoidCallback cb, Object ctx) throws org.apache.zookeeper.KeeperException
      Throws:
      org.apache.zookeeper.KeeperException

    • filterByPrefix

      private static List<String> filterByPrefix(List<String> nodes, String... prefixes)
      Filters the given node list by the given prefixes. This method is all-inclusive--if any element in the node list starts with any of the given prefixes, then it is included in the result.
      Parameters:
      nodes - the nodes to filter
      prefixes - the prefixes to include in the result
      Returns:
      list of every element that starts with one of the prefixes

    • getIdentifier

      public String getIdentifier()