Package org.apache.hadoop.hbase
Class DistributedHBaseCluster
java.lang.Object
org.apache.hadoop.hbase.HBaseClusterInterface
org.apache.hadoop.hbase.DistributedHBaseCluster
- All Implemented Interfaces:
Closeable,AutoCloseable,org.apache.hadoop.conf.Configurable
Manages the interactions with an already deployed distributed cluster (as opposed to a
pseudo-distributed, or mini/local cluster). This is used by integration and system tests.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprivate static class -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate org.apache.hadoop.hbase.client.Adminprivate ClusterManagerprivate final org.apache.hadoop.hbase.client.Connectionprivate final Set<org.apache.hadoop.hbase.ServerName>List of RegionServers killed so far.private static final org.slf4j.LoggerFields inherited from class org.apache.hadoop.hbase.HBaseClusterInterface
conf, initialClusterStatus -
Constructor Summary
ConstructorsConstructorDescriptionDistributedHBaseCluster(org.apache.hadoop.conf.Configuration conf, ClusterManager clusterManager) -
Method Summary
Modifier and TypeMethodDescriptionvoidclose()Closes all the resources held open for this cluster.org.apache.hadoop.hbase.ClusterMetricsReturns a ClusterStatus for this HBase clusterorg.apache.hadoop.hbase.ClusterMetricsReturns a ClusterStatus for this HBase cluster as observed at the starting of the HBaseClusterorg.apache.hadoop.hbase.ServerNamegetServerHoldingRegion(org.apache.hadoop.hbase.TableName tn, byte[] regionName) Get the ServerName of region server serving the specified regionbooleanbooleanisKilledRS(org.apache.hadoop.hbase.ServerName serverName) Keeping track of killed servers and being able to check if a particular server was killed makes it possible to do fault tolerance testing for dead servers in a deterministic way.voidkillDataNode(org.apache.hadoop.hbase.ServerName serverName) Kills the datanode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillJournalNode(org.apache.hadoop.hbase.ServerName serverName) Kills the journalnode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillMaster(org.apache.hadoop.hbase.ServerName serverName) Kills the master process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillNameNode(org.apache.hadoop.hbase.ServerName serverName) Kills the namenode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillRegionServer(org.apache.hadoop.hbase.ServerName serverName) Kills the region server process if this is a distributed cluster, otherwise this causes the region server to exit doing basic clean up only.voidkillZkNode(org.apache.hadoop.hbase.ServerName serverName) Kills the zookeeper node process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.protected booleanbooleanrestoreClusterMetrics(org.apache.hadoop.hbase.ClusterMetrics initial) Restores the cluster to given state if this is a real cluster, otherwise does nothing.protected booleanrestoreMasters(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) protected booleanrestoreRegionServers(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) voidresumeRegionServer(org.apache.hadoop.hbase.ServerName serverName) Resume the region servervoidsetClusterManager(ClusterManager clusterManager) voidshutdown()Shut down the HBase clustervoidstartDataNode(org.apache.hadoop.hbase.ServerName serverName) Starts a new datanode on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstartJournalNode(org.apache.hadoop.hbase.ServerName serverName) Starts a new journalnode on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstartMaster(String hostname, int port) Starts a new master on the given hostname or if this is a mini/local cluster, starts a master locally.voidstartNameNode(org.apache.hadoop.hbase.ServerName serverName) Starts a new namenode on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstartRegionServer(String hostname, int port) Starts a new region server on the given hostname or if this is a mini/local cluster, starts a region server locally.voidstartZkNode(String hostname, int port) Starts a new zookeeper node on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstopDataNode(org.apache.hadoop.hbase.ServerName serverName) Stops the datanode if this is a distributed cluster, otherwise silently logs warning message.voidstopJournalNode(org.apache.hadoop.hbase.ServerName serverName) Stops the journalnode if this is a distributed cluster, otherwise silently logs warning message.voidstopMaster(org.apache.hadoop.hbase.ServerName serverName) Stops the given master, by attempting a gradual stop.voidstopNameNode(org.apache.hadoop.hbase.ServerName serverName) Stops the namenode if this is a distributed cluster, otherwise silently logs warning message.voidstopRegionServer(org.apache.hadoop.hbase.ServerName serverName) Stops the given region server, by attempting a gradual stop.voidstopZkNode(org.apache.hadoop.hbase.ServerName serverName) Stops the region zookeeper if this is a distributed cluster, otherwise silently logs warning message.voidsuspendRegionServer(org.apache.hadoop.hbase.ServerName serverName) Suspend the region serverbooleanwaitForActiveAndReadyMaster(long timeout) Blocks until there is an active master and that master has completed initialization.voidwaitForDataNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified datanode to join the clustervoidwaitForDataNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified datanode to stop the thread / process.voidwaitForJournalNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified journalnode to join the clustervoidwaitForJournalNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified journalnode to stopvoidwaitForMasterToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified master to stop the thread / process.voidwaitForNameNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified namenode to join the clustervoidwaitForNameNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified namenode to stopvoidwaitForRegionServerToResume(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified region server to resume the thread / process.voidwaitForRegionServerToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified region server to stop the thread / process.voidwaitForRegionServerToSuspend(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified region server to suspend the thread / process.private voidwaitForServiceToResume(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) private voidwaitForServiceToStart(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) private voidwaitForServiceToStop(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) private voidwaitForServiceToSuspend(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) voidwaitForZkNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified zookeeper node to join the clustervoidwaitForZkNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified zookeeper node to stop the thread / process.voidWait for HBase Cluster to shut down.Methods inherited from class org.apache.hadoop.hbase.HBaseClusterInterface
getConf, getServerHoldingMeta, restoreInitialStatus, setConf, waitForActiveAndReadyMaster, waitForDatanodesRegistered, waitForNamenodeAvailable, waitForRegionServerToStart
-
Field Details
-
LOG
-
admin
-
connection
-
clusterManager
-
killedRegionServers
List of RegionServers killed so far. ServerName also comprises startCode of a server, so any restarted instances of the same server will have different ServerName and will not coincide with past dead ones. So there's no need to cleanup this list.
-
-
Constructor Details
-
DistributedHBaseCluster
public DistributedHBaseCluster(org.apache.hadoop.conf.Configuration conf, ClusterManager clusterManager) throws IOException - Throws:
IOException
-
-
Method Details
-
setClusterManager
-
getClusterManager
-
getClusterMetrics
Returns a ClusterStatus for this HBase cluster- Specified by:
getClusterMetricsin classHBaseClusterInterface- Throws:
IOException- See Also:
-
getInitialClusterMetrics
Description copied from class:HBaseClusterInterfaceReturns a ClusterStatus for this HBase cluster as observed at the starting of the HBaseCluster- Overrides:
getInitialClusterMetricsin classHBaseClusterInterface- Throws:
IOException
-
close
Description copied from class:HBaseClusterInterfaceCloses all the resources held open for this cluster. Note that this call does not shutdown the cluster.- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Specified by:
closein classHBaseClusterInterface- Throws:
IOException- See Also:
-
startRegionServer
Description copied from class:HBaseClusterInterfaceStarts a new region server on the given hostname or if this is a mini/local cluster, starts a region server locally.- Specified by:
startRegionServerin classHBaseClusterInterface- Parameters:
hostname- the hostname to start the regionserver on- Throws:
IOException- if something goes wrong
-
killRegionServer
Description copied from class:HBaseClusterInterfaceKills the region server process if this is a distributed cluster, otherwise this causes the region server to exit doing basic clean up only.- Specified by:
killRegionServerin classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
isKilledRS
Description copied from class:HBaseClusterInterfaceKeeping track of killed servers and being able to check if a particular server was killed makes it possible to do fault tolerance testing for dead servers in a deterministic way. A concrete example of such case is - killing servers and waiting for all regions of a particular table to be assigned. We can check for server column in META table and that its value is not one of the killed servers.- Specified by:
isKilledRSin classHBaseClusterInterface
-
stopRegionServer
Description copied from class:HBaseClusterInterfaceStops the given region server, by attempting a gradual stop.- Specified by:
stopRegionServerin classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
waitForRegionServerToStop
public void waitForRegionServerToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified region server to stop the thread / process.- Specified by:
waitForRegionServerToStopin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForRegionServerToSuspend
public void waitForRegionServerToSuspend(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified region server to suspend the thread / process.- Specified by:
waitForRegionServerToSuspendin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForRegionServerToResume
public void waitForRegionServerToResume(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified region server to resume the thread / process.- Specified by:
waitForRegionServerToResumein classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
suspendRegionServer
Description copied from class:HBaseClusterInterfaceSuspend the region server- Specified by:
suspendRegionServerin classHBaseClusterInterface- Parameters:
serverName- the hostname to suspend the regionserver on- Throws:
IOException- if something goes wrong
-
resumeRegionServer
Description copied from class:HBaseClusterInterfaceResume the region server- Specified by:
resumeRegionServerin classHBaseClusterInterface- Parameters:
serverName- the hostname to resume the regionserver on- Throws:
IOException- if something goes wrong
-
startZkNode
Description copied from class:HBaseClusterInterfaceStarts a new zookeeper node on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startZkNodein classHBaseClusterInterface- Parameters:
hostname- the hostname to start the regionserver on- Throws:
IOException- if something goes wrong
-
killZkNode
Description copied from class:HBaseClusterInterfaceKills the zookeeper node process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killZkNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
stopZkNode
Description copied from class:HBaseClusterInterfaceStops the region zookeeper if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopZkNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
waitForZkNodeToStart
public void waitForZkNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified zookeeper node to join the cluster- Specified by:
waitForZkNodeToStartin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForZkNodeToStop
public void waitForZkNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified zookeeper node to stop the thread / process.- Specified by:
waitForZkNodeToStopin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
startDataNode
Description copied from class:HBaseClusterInterfaceStarts a new datanode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startDataNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
killDataNode
Description copied from class:HBaseClusterInterfaceKills the datanode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killDataNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
stopDataNode
Description copied from class:HBaseClusterInterfaceStops the datanode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopDataNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
waitForDataNodeToStart
public void waitForDataNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified datanode to join the cluster- Specified by:
waitForDataNodeToStartin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForDataNodeToStop
public void waitForDataNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified datanode to stop the thread / process.- Specified by:
waitForDataNodeToStopin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
startNameNode
Description copied from class:HBaseClusterInterfaceStarts a new namenode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startNameNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
killNameNode
Description copied from class:HBaseClusterInterfaceKills the namenode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killNameNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
stopNameNode
Description copied from class:HBaseClusterInterfaceStops the namenode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopNameNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
waitForNameNodeToStart
public void waitForNameNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified namenode to join the cluster- Specified by:
waitForNameNodeToStartin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForNameNodeToStop
public void waitForNameNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified namenode to stop- Specified by:
waitForNameNodeToStopin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
startJournalNode
Description copied from class:HBaseClusterInterfaceStarts a new journalnode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startJournalNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
killJournalNode
Description copied from class:HBaseClusterInterfaceKills the journalnode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killJournalNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
stopJournalNode
Description copied from class:HBaseClusterInterfaceStops the journalnode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopJournalNodein classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
waitForJournalNodeToStart
public void waitForJournalNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified journalnode to join the cluster- Specified by:
waitForJournalNodeToStartin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForJournalNodeToStop
public void waitForJournalNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified journalnode to stop- Specified by:
waitForJournalNodeToStopin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForServiceToStop
private void waitForServiceToStop(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
waitForServiceToStart
private void waitForServiceToStart(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
waitForServiceToSuspend
private void waitForServiceToSuspend(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
waitForServiceToResume
private void waitForServiceToResume(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
startMaster
Description copied from class:HBaseClusterInterfaceStarts a new master on the given hostname or if this is a mini/local cluster, starts a master locally.- Specified by:
startMasterin classHBaseClusterInterface- Parameters:
hostname- the hostname to start the master on- Throws:
IOException- if something goes wrong
-
killMaster
Description copied from class:HBaseClusterInterfaceKills the master process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killMasterin classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
stopMaster
Description copied from class:HBaseClusterInterfaceStops the given master, by attempting a gradual stop.- Specified by:
stopMasterin classHBaseClusterInterface- Throws:
IOException- if something goes wrong
-
waitForMasterToStop
public void waitForMasterToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterInterfaceWait for the specified master to stop the thread / process.- Specified by:
waitForMasterToStopin classHBaseClusterInterface- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForActiveAndReadyMaster
Description copied from class:HBaseClusterInterfaceBlocks until there is an active master and that master has completed initialization.- Specified by:
waitForActiveAndReadyMasterin classHBaseClusterInterface- Parameters:
timeout- the timeout limit in ms- Returns:
- true if an active master becomes available. false if there are no masters left.
- Throws:
IOException
-
getServerHoldingRegion
public org.apache.hadoop.hbase.ServerName getServerHoldingRegion(org.apache.hadoop.hbase.TableName tn, byte[] regionName) throws IOException Description copied from class:HBaseClusterInterfaceGet the ServerName of region server serving the specified region- Specified by:
getServerHoldingRegionin classHBaseClusterInterface- Parameters:
tn- Table name that has the region.regionName- Name of the region in bytes- Returns:
- ServerName that hosts the region or null
- Throws:
IOException
-
waitUntilShutDown
Description copied from class:HBaseClusterInterfaceWait for HBase Cluster to shut down.- Specified by:
waitUntilShutDownin classHBaseClusterInterface
-
shutdown
Description copied from class:HBaseClusterInterfaceShut down the HBase cluster- Specified by:
shutdownin classHBaseClusterInterface- Throws:
IOException
-
isDistributedCluster
- Overrides:
isDistributedClusterin classHBaseClusterInterface- Returns:
- whether we are interacting with a distributed cluster as opposed to an in-process mini/local cluster.
-
restoreClusterMetrics
public boolean restoreClusterMetrics(org.apache.hadoop.hbase.ClusterMetrics initial) throws IOException Description copied from class:HBaseClusterInterfaceRestores the cluster to given state if this is a real cluster, otherwise does nothing. This is a best effort restore. If the servers are not reachable, or insufficient permissions, etc. restoration might be partial.- Overrides:
restoreClusterMetricsin classHBaseClusterInterface- Returns:
- whether restoration is complete
- Throws:
IOException
-
restoreMasters
protected boolean restoreMasters(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) -
restoreRegionServers
protected boolean restoreRegionServers(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) -
restoreAdmin
- Throws:
IOException
-