Package org.apache.hadoop.hbase
Class DistributedHBaseCluster
java.lang.Object
org.apache.hadoop.hbase.HBaseCluster
org.apache.hadoop.hbase.DistributedHBaseCluster
- All Implemented Interfaces:
Closeable,AutoCloseable,org.apache.hadoop.conf.Configurable
Manages the interactions with an already deployed distributed cluster (as opposed to a
pseudo-distributed, or mini/local cluster). This is used by integration and system tests.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprivate static class -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate org.apache.hadoop.hbase.client.Adminprivate ClusterManagerprivate final org.apache.hadoop.hbase.client.Connectionprivate final Set<org.apache.hadoop.hbase.ServerName>List of RegionServers killed so far.Fields inherited from class org.apache.hadoop.hbase.HBaseCluster
conf, initialClusterStatus, LOG -
Constructor Summary
ConstructorsConstructorDescriptionDistributedHBaseCluster(org.apache.hadoop.conf.Configuration conf, ClusterManager clusterManager) -
Method Summary
Modifier and TypeMethodDescriptionvoidclose()Closes all the resources held open for this cluster.org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterfacegetAdminProtocol(org.apache.hadoop.hbase.ServerName serverName) Returns an AdminProtocol interface to the regionserverorg.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService.BlockingInterfacegetClientProtocol(org.apache.hadoop.hbase.ServerName serverName) Returns a ClientProtocol interface to the regionserverorg.apache.hadoop.hbase.ClusterMetricsReturns a ClusterStatus for this HBase clusterorg.apache.hadoop.hbase.ClusterMetricsReturns a ClusterStatus for this HBase cluster as observed at the starting of the HBaseClusterorg.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterService.BlockingInterfaceReturns anMasterProtos.MasterService.BlockingInterfaceto the active masterorg.apache.hadoop.hbase.ServerNamegetServerHoldingRegion(org.apache.hadoop.hbase.TableName tn, byte[] regionName) Get the ServerName of region server serving the specified regionbooleanbooleanisKilledRS(org.apache.hadoop.hbase.ServerName serverName) Keeping track of killed servers and being able to check if a particular server was killed makes it possible to do fault tolerance testing for dead servers in a deterministic way.voidkillDataNode(org.apache.hadoop.hbase.ServerName serverName) Kills the datanode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillJournalNode(org.apache.hadoop.hbase.ServerName serverName) Kills the journalnode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillMaster(org.apache.hadoop.hbase.ServerName serverName) Kills the master process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillNameNode(org.apache.hadoop.hbase.ServerName serverName) Kills the namenode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.voidkillRegionServer(org.apache.hadoop.hbase.ServerName serverName) Kills the region server process if this is a distributed cluster, otherwise this causes the region server to exit doing basic clean up only.voidkillZkNode(org.apache.hadoop.hbase.ServerName serverName) Kills the zookeeper node process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.protected booleanbooleanrestoreClusterMetrics(org.apache.hadoop.hbase.ClusterMetrics initial) Restores the cluster to given state if this is a real cluster, otherwise does nothing.protected booleanrestoreMasters(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) protected booleanrestoreRegionServers(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) voidresumeRegionServer(org.apache.hadoop.hbase.ServerName serverName) Resume the region servervoidsetClusterManager(ClusterManager clusterManager) voidshutdown()Shut down the HBase clustervoidstartDataNode(org.apache.hadoop.hbase.ServerName serverName) Starts a new datanode on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstartJournalNode(org.apache.hadoop.hbase.ServerName serverName) Starts a new journalnode on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstartMaster(String hostname, int port) Starts a new master on the given hostname or if this is a mini/local cluster, starts a master locally.voidstartNameNode(org.apache.hadoop.hbase.ServerName serverName) Starts a new namenode on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstartRegionServer(String hostname, int port) Starts a new region server on the given hostname or if this is a mini/local cluster, starts a region server locally.voidstartZkNode(String hostname, int port) Starts a new zookeeper node on the given hostname or if this is a mini/local cluster, silently logs warning message.voidstopDataNode(org.apache.hadoop.hbase.ServerName serverName) Stops the datanode if this is a distributed cluster, otherwise silently logs warning message.voidstopJournalNode(org.apache.hadoop.hbase.ServerName serverName) Stops the journalnode if this is a distributed cluster, otherwise silently logs warning message.voidstopMaster(org.apache.hadoop.hbase.ServerName serverName) Stops the given master, by attempting a gradual stop.voidstopNameNode(org.apache.hadoop.hbase.ServerName serverName) Stops the namenode if this is a distributed cluster, otherwise silently logs warning message.voidstopRegionServer(org.apache.hadoop.hbase.ServerName serverName) Stops the given region server, by attempting a gradual stop.voidstopZkNode(org.apache.hadoop.hbase.ServerName serverName) Stops the region zookeeper if this is a distributed cluster, otherwise silently logs warning message.voidsuspendRegionServer(org.apache.hadoop.hbase.ServerName serverName) Suspend the region serverbooleanwaitForActiveAndReadyMaster(long timeout) Blocks until there is an active master and that master has completed initialization.voidwaitForDataNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified datanode to join the clustervoidwaitForDataNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified datanode to stop the thread / process.voidwaitForJournalNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified journalnode to join the clustervoidwaitForJournalNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified journalnode to stopvoidwaitForMasterToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified master to stop the thread / process.voidwaitForNameNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified namenode to join the clustervoidwaitForNameNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified namenode to stopvoidwaitForRegionServerToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified region server to stop the thread / process.private voidwaitForServiceToStart(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) private voidwaitForServiceToStop(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) voidwaitForZkNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified zookeeper node to join the clustervoidwaitForZkNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified zookeeper node to stop the thread / process.voidWait for HBase Cluster to shut down.Methods inherited from class org.apache.hadoop.hbase.HBaseCluster
getConf, getServerHoldingMeta, restoreInitialStatus, setConf, waitForActiveAndReadyMaster, waitForDatanodesRegistered, waitForNamenodeAvailable, waitForRegionServerToStart
-
Field Details
-
admin
-
connection
-
clusterManager
-
killedRegionServers
List of RegionServers killed so far. ServerName also comprises startCode of a server, so any restarted instances of the same server will have different ServerName and will not coincide with past dead ones. So there's no need to cleanup this list.
-
-
Constructor Details
-
DistributedHBaseCluster
public DistributedHBaseCluster(org.apache.hadoop.conf.Configuration conf, ClusterManager clusterManager) throws IOException - Throws:
IOException
-
-
Method Details
-
setClusterManager
-
getClusterManager
-
getClusterMetrics
Returns a ClusterStatus for this HBase cluster- Specified by:
getClusterMetricsin classHBaseCluster- Throws:
IOException- See Also:
-
getInitialClusterMetrics
Description copied from class:HBaseClusterReturns a ClusterStatus for this HBase cluster as observed at the starting of the HBaseCluster- Overrides:
getInitialClusterMetricsin classHBaseCluster- Throws:
IOException
-
close
Description copied from class:HBaseClusterCloses all the resources held open for this cluster. Note that this call does not shutdown the cluster.- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Specified by:
closein classHBaseCluster- Throws:
IOException- See Also:
-
getAdminProtocol
public org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface getAdminProtocol(org.apache.hadoop.hbase.ServerName serverName) throws IOException Description copied from class:HBaseClusterReturns an AdminProtocol interface to the regionserver- Specified by:
getAdminProtocolin classHBaseCluster- Throws:
IOException
-
getClientProtocol
public org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService.BlockingInterface getClientProtocol(org.apache.hadoop.hbase.ServerName serverName) throws IOException Description copied from class:HBaseClusterReturns a ClientProtocol interface to the regionserver- Specified by:
getClientProtocolin classHBaseCluster- Throws:
IOException
-
startRegionServer
Description copied from class:HBaseClusterStarts a new region server on the given hostname or if this is a mini/local cluster, starts a region server locally.- Specified by:
startRegionServerin classHBaseCluster- Parameters:
hostname- the hostname to start the regionserver on- Throws:
IOException- if something goes wrong
-
killRegionServer
Description copied from class:HBaseClusterKills the region server process if this is a distributed cluster, otherwise this causes the region server to exit doing basic clean up only.- Specified by:
killRegionServerin classHBaseCluster- Throws:
IOException- if something goes wrong
-
isKilledRS
Description copied from class:HBaseClusterKeeping track of killed servers and being able to check if a particular server was killed makes it possible to do fault tolerance testing for dead servers in a deterministic way. A concrete example of such case is - killing servers and waiting for all regions of a particular table to be assigned. We can check for server column in META table and that its value is not one of the killed servers.- Specified by:
isKilledRSin classHBaseCluster
-
stopRegionServer
Description copied from class:HBaseClusterStops the given region server, by attempting a gradual stop.- Specified by:
stopRegionServerin classHBaseCluster- Throws:
IOException- if something goes wrong
-
waitForRegionServerToStop
public void waitForRegionServerToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified region server to stop the thread / process.- Specified by:
waitForRegionServerToStopin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
suspendRegionServer
Description copied from class:HBaseClusterSuspend the region server- Specified by:
suspendRegionServerin classHBaseCluster- Parameters:
serverName- the hostname to suspend the regionserver on- Throws:
IOException- if something goes wrong
-
resumeRegionServer
Description copied from class:HBaseClusterResume the region server- Specified by:
resumeRegionServerin classHBaseCluster- Parameters:
serverName- the hostname to resume the regionserver on- Throws:
IOException- if something goes wrong
-
startZkNode
Description copied from class:HBaseClusterStarts a new zookeeper node on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startZkNodein classHBaseCluster- Parameters:
hostname- the hostname to start the regionserver on- Throws:
IOException- if something goes wrong
-
killZkNode
Description copied from class:HBaseClusterKills the zookeeper node process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killZkNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
stopZkNode
Description copied from class:HBaseClusterStops the region zookeeper if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopZkNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
waitForZkNodeToStart
public void waitForZkNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified zookeeper node to join the cluster- Specified by:
waitForZkNodeToStartin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForZkNodeToStop
public void waitForZkNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified zookeeper node to stop the thread / process.- Specified by:
waitForZkNodeToStopin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
startDataNode
Description copied from class:HBaseClusterStarts a new datanode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startDataNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
killDataNode
Description copied from class:HBaseClusterKills the datanode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killDataNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
stopDataNode
Description copied from class:HBaseClusterStops the datanode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopDataNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
waitForDataNodeToStart
public void waitForDataNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified datanode to join the cluster- Specified by:
waitForDataNodeToStartin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForDataNodeToStop
public void waitForDataNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified datanode to stop the thread / process.- Specified by:
waitForDataNodeToStopin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
startNameNode
Description copied from class:HBaseClusterStarts a new namenode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startNameNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
killNameNode
Description copied from class:HBaseClusterKills the namenode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killNameNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
stopNameNode
Description copied from class:HBaseClusterStops the namenode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopNameNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
waitForNameNodeToStart
public void waitForNameNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified namenode to join the cluster- Specified by:
waitForNameNodeToStartin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForNameNodeToStop
public void waitForNameNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified namenode to stop- Specified by:
waitForNameNodeToStopin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
startJournalNode
Description copied from class:HBaseClusterStarts a new journalnode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startJournalNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
killJournalNode
Description copied from class:HBaseClusterKills the journalnode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killJournalNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
stopJournalNode
Description copied from class:HBaseClusterStops the journalnode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopJournalNodein classHBaseCluster- Throws:
IOException- if something goes wrong
-
waitForJournalNodeToStart
public void waitForJournalNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified journalnode to join the cluster- Specified by:
waitForJournalNodeToStartin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForJournalNodeToStop
public void waitForJournalNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified journalnode to stop- Specified by:
waitForJournalNodeToStopin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForServiceToStop
private void waitForServiceToStop(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
waitForServiceToStart
private void waitForServiceToStart(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
getMasterAdminService
public org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterService.BlockingInterface getMasterAdminService() throws IOExceptionDescription copied from class:HBaseClusterReturns anMasterProtos.MasterService.BlockingInterfaceto the active master- Specified by:
getMasterAdminServicein classHBaseCluster- Throws:
IOException
-
startMaster
Description copied from class:HBaseClusterStarts a new master on the given hostname or if this is a mini/local cluster, starts a master locally.- Specified by:
startMasterin classHBaseCluster- Parameters:
hostname- the hostname to start the master on- Throws:
IOException- if something goes wrong
-
killMaster
Description copied from class:HBaseClusterKills the master process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killMasterin classHBaseCluster- Throws:
IOException- if something goes wrong
-
stopMaster
Description copied from class:HBaseClusterStops the given master, by attempting a gradual stop.- Specified by:
stopMasterin classHBaseCluster- Throws:
IOException- if something goes wrong
-
waitForMasterToStop
public void waitForMasterToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseClusterWait for the specified master to stop the thread / process.- Specified by:
waitForMasterToStopin classHBaseCluster- Throws:
IOException- if something goes wrong or timeout occurs
-
waitForActiveAndReadyMaster
Description copied from class:HBaseClusterBlocks until there is an active master and that master has completed initialization.- Specified by:
waitForActiveAndReadyMasterin classHBaseCluster- Parameters:
timeout- the timeout limit in ms- Returns:
- true if an active master becomes available. false if there are no masters left.
- Throws:
IOException
-
getServerHoldingRegion
public org.apache.hadoop.hbase.ServerName getServerHoldingRegion(org.apache.hadoop.hbase.TableName tn, byte[] regionName) throws IOException Description copied from class:HBaseClusterGet the ServerName of region server serving the specified region- Specified by:
getServerHoldingRegionin classHBaseCluster- Parameters:
tn- Table name that has the region.regionName- Name of the region in bytes- Returns:
- ServerName that hosts the region or null
- Throws:
IOException
-
waitUntilShutDown
Description copied from class:HBaseClusterWait for HBase Cluster to shut down.- Specified by:
waitUntilShutDownin classHBaseCluster
-
shutdown
Description copied from class:HBaseClusterShut down the HBase cluster- Specified by:
shutdownin classHBaseCluster- Throws:
IOException
-
isDistributedCluster
- Overrides:
isDistributedClusterin classHBaseCluster- Returns:
- whether we are interacting with a distributed cluster as opposed to an in-process mini/local cluster.
-
restoreClusterMetrics
public boolean restoreClusterMetrics(org.apache.hadoop.hbase.ClusterMetrics initial) throws IOException Description copied from class:HBaseClusterRestores the cluster to given state if this is a real cluster, otherwise does nothing. This is a best effort restore. If the servers are not reachable, or insufficient permissions, etc. restoration might be partial.- Overrides:
restoreClusterMetricsin classHBaseCluster- Returns:
- whether restoration is complete
- Throws:
IOException
-
restoreMasters
protected boolean restoreMasters(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) -
restoreRegionServers
protected boolean restoreRegionServers(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) -
restoreAdmin
- Throws:
IOException
-