Package org.apache.hadoop.hbase.master

Class HMaster

java.lang.Object

java.lang.Thread

org.apache.hadoop.hbase.HBaseServerBase<MasterRpcServices>

org.apache.hadoop.hbase.master.HMaster

All Implemented Interfaces:

Runnable, Abortable, ConnectionRegistryEndpoint, ConfigurationObserver, MasterServices, Server, Stoppable

Direct Known Subclasses:

HMasterCommandLine.LocalHMaster

@LimitedPrivate("Tools")
public class HMaster
extends HBaseServerBase<MasterRpcServices>
implements MasterServices

HMaster is the "master server" for HBase. An HBase cluster has one active master. If many masters are started, all compete. Whichever wins goes on to run the cluster. All others park themselves in their constructor until master or cluster shutdown or until the active master loses its lease in zookeeper. Thereafter, all running master jostle to take over master role.

The Master can be asked shutdown the cluster. See shutdown(). In this case it will tell all regionservers to go down and then wait on them all reporting in that they are down. This master will then shut itself down.

You can also shutdown just this master. Call stopMaster().

See Also:

• Watcher

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static interface	HMaster.TableDescriptorGetter	Implement to return TableDescriptor after pre-checks

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields

Modifier and Type	Field	Description
private boolean	activeMaster
private final ActiveMasterManager	activeMasterManager
private AssignmentManager	assignmentManager
private RSGroupBasedLoadBalancer	balancer
private BalancerChore	balancerChore
private final CachedClusterId	cachedClusterId
(package private) CatalogJanitor	catalogJanitorChore
private String	clusterId
private ClusterSchemaService	clusterSchemaService
private ClusterStatusChore	clusterStatusChore
private ClusterStatusPublisher	clusterStatusPublisherChore
(package private) Map<String,org.apache.hbase.thirdparty.com.google.protobuf.Service>	coprocessorServiceHandlers
(package private) MasterCoprocessorHost	cpHost
static final int	DEFAULT_HBASE_MASTER_CLEANER_INTERVAL
private static final String	DEFAULT_HBASE_MASTER_RSPROC_DISPATCHER_CLASS
static final int	DEFAULT_HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS
private static final boolean	DEFAULT_WARMUP_BEFORE_MOVE
private static boolean	disableBalancerChoreForTest
private DrainingServerTracker	drainingServerTracker
private DirScanPool	exclusiveHFileCleanerPool
private MasterFileSystem	fileSystemManager
static final String	HBASE_MASTER_CLEANER_INTERVAL
static final String	HBASE_MASTER_RSPROC_DISPATCHER_CLASS	Use RSProcedureDispatcher instance to initiate master -> rs remote procedure execution.
static final String	HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS
private HbckChore	hbckChore
private List<org.apache.hadoop.fs.Path>	hfileCleanerPaths
private List<HFileCleaner>	hfileCleaners
private final ProcedureEvent<?>	initialized
(package private) LoadBalancerStateStore	loadBalancerStateStore
private final LockManager	lockManager
private static final org.slf4j.Logger	LOG
private LogCleaner	logCleaner
private DirScanPool	logCleanerPool
(package private) static final String	MAINTENANCE_MODE
private final boolean	maintenanceMode
private JVMClusterUtil.RegionServerThread	maintenanceRegionServer
static final String	MASTER
private long	masterActiveTime
(package private) MasterAddressSyncer	masterAddressSyncer
private long	masterFinishedInitializationTime
private org.apache.hbase.thirdparty.org.eclipse.jetty.server.Server	masterJettyServer	jetty server for master to redirect requests to regionserver infoServer
private MasterRegion	masterRegion
private final int	maxBalancingTime
private final double	maxRitPercent
private MetaLocationSyncer	metaLocationSyncer
(package private) final MetricsMaster	metricsMaster
private final IdLock	mobCompactionLock
private Map<TableName,AtomicInteger>	mobCompactionStates
private MobFileCleanerChore	mobFileCleanerChore
private MobFileCompactionChore	mobFileCompactionChore
private MasterProcedureManagerHost	mpmHost
private OldWALsDirSizeChore	oldWALsDirSizeChore
private final boolean	preLoadTableDescriptors
private ProcedureExecutor<MasterProcedureEnv>	procedureExecutor
private ProcedureStore	procedureStore
private MasterQuotaManager	quotaManager
private QuotaObserverChore	quotaObserverChore
private RegionNormalizerManager	regionNormalizerManager
private final RegionServerTracker	regionServerTracker
private RegionsRecoveryChore	regionsRecoveryChore
private RegionsRecoveryConfigManager	regionsRecoveryConfigManager
private ReplicationBarrierCleaner	replicationBarrierCleaner
private final ReplicationLogCleanerBarrier	replicationLogCleanerBarrier
private ReplicationPeerManager	replicationPeerManager
private ReplicationPeerModificationStateStore	replicationPeerModificationStateStore	Store whether we allow replication peer modification operations.
private RollingUpgradeChore	rollingUpgradeChore
(package private) MemoryBoundedLogMessageBuffer	rsFatals
private RSGroupInfoManager	rsGroupInfoManager
private RegionServerList	rsListStorage
private ServerManager	serverManager
(package private) boolean	serviceStarted
private DirScanPool	sharedHFileCleanerPool
private SnapshotCleanerChore	snapshotCleanerChore
(package private) SnapshotCleanupStateStore	snapshotCleanupStateStore
(package private) SnapshotManager	snapshotManager
private SnapshotQuotaObserverChore	snapshotQuotaChore
private SpaceQuotaSnapshotNotifier	spaceQuotaSnapshotNotifier
private SplitOrMergeStateStore	splitOrMergeStateStore
private SplitWALManager	splitWALManager
private TaskGroup	startupTaskGroup
private final Semaphore	syncReplicationPeerLock
private SyncReplicationReplayWALManager	syncReplicationReplayWALManager
private TableStateManager	tableStateManager
private MasterWalManager	walManager
static final String	WARMUP_BEFORE_MOVE

Fields inherited from class org.apache.hadoop.hbase.HBaseServerBase

abortRequested, asyncClusterConnection, choreService, clusterStatusTracker, conf, configurationManager, csm, dataFs, dataRootDir, eventLoopGroupConfig, executorService, infoServer, metaRegionLocationCache, msgInterval, namedQueueRecorder, rpcServices, serverName, sleeper, startcode, stopped, tableDescriptors, userProvider, useThisHostnameInstead, walFs, walRootDir, zooKeeper

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor	Description
HMaster(org.apache.hadoop.conf.Configuration conf)	Initializes the HMaster.

Method Summary

Modifier and Type	Method	Description
void	abort(String reason, Throwable cause)	Abort the server or client.
boolean	abortProcedure(long procId, boolean mayInterruptIfRunning)	Abort a procedure.
long	addColumn(TableName tableName, ColumnFamilyDescriptor column, long nonceGroup, long nonce)	Add a new column to an existing table
long	addReplicationPeer(String peerId, ReplicationPeerConfig peerConfig, boolean enabled)	Add a new replication peer for replicating data to slave cluster
BalanceResponse	balance()
BalanceResponse	balance(BalanceRequest request)
BalanceResponse	balanceOrUpdateMetrics()	Trigger a normal balance, see balance() .
boolean	balanceSwitch(boolean b)
private void	balanceThrottling(long nextBalanceStartTime, int maxRegionsInTransition, long cutoffTime)	It first sleep to the next balance plan start time.
private org.apache.hbase.thirdparty.org.eclipse.jetty.ee8.servlet.ServletHolder	buildApiV1Servlet()
private org.apache.hbase.thirdparty.org.eclipse.jetty.ee8.servlet.ServletHolder	buildHbckServlet()
protected boolean	cacheTableDescriptor()
protected boolean	canCreateBaseZNode()
protected boolean	canUpdateTableDescriptor()
void	checkIfShouldMoveSystemRegionAsync()	Called when a new RegionServer is added to the cluster.
(package private) void	checkInitialized()
protected void	checkServiceStarted()
private void	checkTableExists(TableName tableName)
void	checkTableModifiable(TableName tableName)	Check table is modifiable; i.e.
protected boolean	clusterMode()
protected void	configureInfoServer(InfoServer infoServer)
static HMaster	constructMaster(Class<? extends HMaster> masterClass, org.apache.hadoop.conf.Configuration conf)	Utility for constructing an instance of the passed HMaster class.
protected ActiveMasterManager	createActiveMasterManager(ZKWatcher zk, ServerName sn, Server server)	Protected to have custom implementations in tests override the default ActiveMaster implementation.
protected AssignmentManager	createAssignmentManager(MasterServices master, MasterRegion masterRegion)
private void	createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
protected NamedQueueRecorder	createNamedQueueRecord()
(package private) long	createNamespace(NamespaceDescriptor namespaceDescriptor, long nonceGroup, long nonce)	Create a new Namespace.
private void	createProcedureExecutor()
private SpaceQuotaSnapshotNotifier	createQuotaSnapshotNotifier()
protected MasterRpcServices	createRpcServices()
protected ServerManager	createServerManager(MasterServices master, RegionServerList storage)	Create a ServerManager instance.
long	createSystemTable(TableDescriptor tableDescriptor)	Create a system table using the given table definition.
long	createTable(TableDescriptor tableDescriptor, byte[][] splitKeys, long nonceGroup, long nonce)	Create a table using the given table definition.
void	decommissionRegionServers(List<ServerName> servers, boolean offload)	Mark region server(s) as decommissioned (previously called 'draining') to prevent additional regions from getting assigned to them.
static void	decorateMasterConfiguration(org.apache.hadoop.conf.Configuration conf)	This method modifies the master's configuration in order to inject replication-related features
long	deleteColumn(TableName tableName, byte[] columnName, long nonceGroup, long nonce)	Delete a column from an existing table
(package private) long	deleteNamespace(String name, long nonceGroup, long nonce)	Delete an existing Namespace.
long	deleteTable(TableName tableName, long nonceGroup, long nonce)	Delete a table
long	disableReplicationPeer(String peerId)	Stop the replication stream to the specified peer
long	disableTable(TableName tableName, long nonceGroup, long nonce)	Disable an existing table
long	enableReplicationPeer(String peerId)	Restart the replication stream to the specified peer
long	enableTable(TableName tableName, long nonceGroup, long nonce)	Enable an existing table
private long	executePeerProcedure(AbstractPeerProcedure<?> procedure)
List<RegionPlan>	executeRegionPlansWithThrottling(List<RegionPlan> plans)	Execute region plans with throttling
private static void	filterTablesByRegex(Collection<TableDescriptor> descriptors, Pattern pattern)	Removes the table descriptors that don't match the pattern.
private void	finishActiveMasterInitialization()	Finish initialization of HMaster after becoming the primary master.
void	flushMasterStore()	Flush master local region
long	flushTable(TableName tableName, List<byte[]> columnFamilies, long nonceGroup, long nonce)	Flush an existing table
Optional<ServerName>	getActiveMaster()	Get active master address.
int	getActiveMasterInfoPort()	Returns info port of active master or 0 if any exception occurs.
ActiveMasterManager	getActiveMasterManager()
AssignmentManager	getAssignmentManager()	Returns Master's instance of the AssignmentManager
double	getAverageLoad()	Compute the average load across all region servers.
int	getBackupMasterInfoPort(ServerName sn)
List<ServerName>	getBackupMasters()	Get backup masters address.
Iterator<ServerName>	getBootstrapNodes()	Get a iterator of the available bootstrap nodes.
CatalogJanitor	getCatalogJanitor()	Returns Master's instance of CatalogJanitor
String	getClientIdAuditPrefix()	Returns Client info for use as prefix on an audit log string; who did an action
String	getClusterId()	Get cluster id.
ClusterMetrics	getClusterMetrics()	Returns cluster status
ClusterMetrics	getClusterMetrics(EnumSet<ClusterMetrics.Option> options)
ClusterMetrics	getClusterMetricsWithoutCoprocessor()
ClusterMetrics	getClusterMetricsWithoutCoprocessor(EnumSet<ClusterMetrics.Option> options)
ClusterSchema	getClusterSchema()	Returns Master's instance of ClusterSchema
CompactionState	getCompactionState(TableName tableName)	Get the compaction state of the table
protected MasterCoprocessorHost	getCoprocessorHost()
protected Class<? extends javax.servlet.http.HttpServlet>	getDumpServlet()
FavoredNodesManager	getFavoredNodesManager()	Returns Favored Nodes Manager
HbckChore	getHbckChore()	Returns Master's instance of HbckChore
HFileCleaner	getHFileCleaner()
List<HFileCleaner>	getHFileCleaners()
ProcedureEvent<?>	getInitializedEvent()	Returns Tripped when Master has finished initialization.
long	getLastMajorCompactionTimestamp(TableName table)
long	getLastMajorCompactionTimestampForRegion(byte[] regionName)	Returns the timestamp of the last successful major compaction for the passed region or 0 if no HFile resulting from a major compaction exists
Collection<ServerName>	getLiveRegionServers()
RSGroupBasedLoadBalancer	getLoadBalancer()	Returns load balancer
String	getLoadBalancerClassName()	Fetch the configured LoadBalancer class name.
static String	getLoadedCoprocessors()	The set of loaded coprocessors is stored in a static set.
LockManager	getLockManager()	Returns LockManager to lock namespaces/tables/regions.
List<LockedResource>	getLocks()	Get locks
LogCleaner	getLogCleaner()
long	getMasterActiveTime()	Returns timestamp in millis when HMaster became the active master.
MasterCoprocessorHost	getMasterCoprocessorHost()	Returns Master's instance of MasterCoprocessorHost
String[]	getMasterCoprocessors()	Returns array of coprocessor SimpleNames.
MasterFileSystem	getMasterFileSystem()	Returns Master's filesystem MasterFileSystem utility class.
long	getMasterFinishedInitializationTime()	Returns timestamp in millis when HMaster finished becoming the active master
MetricsMaster	getMasterMetrics()	Returns Master's instance of MetricsMaster
ProcedureExecutor<MasterProcedureEnv>	getMasterProcedureExecutor()	Returns Master's instance of ProcedureExecutor
MasterProcedureManagerHost	getMasterProcedureManagerHost()	Returns the underlying MasterProcedureManagerHost
MasterQuotaManager	getMasterQuotaManager()	Returns Master's instance of MasterQuotaManager
MasterRegion	getMasterRegion()
MasterRpcServices	getMasterRpcServices()
long	getMasterStartTime()	Returns timestamp in millis when HMaster was started.
MasterWalManager	getMasterWalManager()	Returns Master's WALs MasterWalManager utility class.
private int	getMaxBalancingTime()	Returns Maximum time we should run balancer for
private int	getMaxRegionsInTransition()	Returns Maximum number of regions in transition
List<HRegionLocation>	getMetaLocations()	Get the location of meta regions.
MetaLocationSyncer	getMetaLocationSyncer()	Get the meta location syncer.
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.GetRegionInfoResponse.CompactionState	getMobCompactionState(TableName tableName)	Gets the mob file compaction state for a specific table.
(package private) NamespaceDescriptor	getNamespace(String name)	Get a Namespace
(package private) List<NamespaceDescriptor>	getNamespaces()	Get all Namespaces
int	getNumWALFiles()
private Map<ServerName,ServerMetrics>	getOnlineServers()
List<Procedure<?>>	getProcedures()	Get procedures
ProcedureStore	getProcedureStore()
protected String	getProcessName()
QuotaObserverChore	getQuotaObserverChore()
RegionNormalizerManager	getRegionNormalizerManager()	Returns Master's instance of RegionNormalizerManager
MemoryBoundedLogMessageBuffer	getRegionServerFatalLogBuffer()
int	getRegionServerInfoPort(ServerName sn)
String	getRegionServerVersion(ServerName sn)
(package private) InetAddress	getRemoteInetAddress(int port, long serverStartCode)	Returns Get remote side's InetAddress
private RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,?>	getRemoteProcedure(long procId)
HashMap<String,List<Pair<ServerName,ReplicationLoadSource>>>	getReplicationLoad(ServerName[] serverNames)
ReplicationLogCleanerBarrier	getReplicationLogCleanerBarrier()	Returns the ReplicationLogCleanerBarrier.
ReplicationPeerConfig	getReplicationPeerConfig(String peerId)	Returns the configured ReplicationPeerConfig for the specified peer
ReplicationPeerManager	getReplicationPeerManager()	Returns the ReplicationPeerManager.
RSGroupInfoManager	getRSGroupInfoManager()	Returns the RSGroupInfoManager
ServerManager	getServerManager()	Returns Master's ServerManager instance.
ServerName	getServerName()	Returns The unique server name for this server.
SnapshotManager	getSnapshotManager()	Returns the underlying snapshot manager
SnapshotQuotaObserverChore	getSnapshotQuotaObserverChore()
SpaceQuotaSnapshotNotifier	getSpaceQuotaSnapshotNotifier()
SplitOrMergeStateStore	getSplitOrMergeStateStore()
SplitWALManager	getSplitWALManager()	Returns return null if current is zk-based WAL splitting
TaskGroup	getStartupProgress()
Semaphore	getSyncReplicationPeerLock()	Returns the SyncReplicationPeerLock.
SyncReplicationReplayWALManager	getSyncReplicationReplayWALManager()	Returns the SyncReplicationReplayWALManager.
private List<TableDescriptor>	getTableDescriptors(List<TableDescriptor> htds, String namespace, String regex, List<TableName> tableNameList, boolean includeSysTables)	Return a list of table table descriptors after applying any provided filter parameters.
TableStateManager	getTableStateManager()	Returns Master's instance of TableStateManager
private List<ServerName>	getUnknownServers()
protected String	getUseThisHostnameInstead(org.apache.hadoop.conf.Configuration conf)
protected void	initClusterSchemaService()
private void	initializeCoprocessorHost(org.apache.hadoop.conf.Configuration conf)
private void	initializeZKBasedSystemTrackers()	Initialize all ZK based system trackers.
private void	initMobCleaner()
private void	initQuotaManager()
boolean	isActiveMaster()	Report whether this master is currently the active master or not.
boolean	isBalancerOn()	Queries the state of the LoadBalancerStateStore.
boolean	isCatalogJanitorEnabled()
private static boolean	isCatalogTable(TableName tableName)
(package private) boolean	isCleanerChoreEnabled()
boolean	isInitialized()	Report whether this master has completed with its initialization and is ready.
boolean	isInMaintenanceMode()	Report whether this master is in maintenance mode.
boolean	isNormalizerOn()	Queries the state of the RegionNormalizerStateStore.
boolean	isOnline()	Report whether this master is started This method is used for testing.
private boolean	isRegionOnline(RegionInfo ri)
boolean	isReplicationPeerModificationEnabled()
boolean	isSplitOrMergeEnabled(MasterSwitchType switchType)	Queries the state of the SplitOrMergeStateStore.
List<ServerName>	listDecommissionedRegionServers()	List region servers marked as decommissioned (previously called 'draining') to not get regions assigned to them.
List<String>	listNamespaces()	List namespace names
List<ReplicationPeerDescription>	listReplicationPeers(String regex)	Return a list of replication peers.
List<TableDescriptor>	listTableDescriptors(String namespace, String regex, List<TableName> tableNameList, boolean includeSysTables)	Returns the list of table descriptors that match the specified request
List<TableDescriptor>	listTableDescriptorsByNamespace(String name)	Get list of table descriptors by namespace
List<TableName>	listTableNames(String namespace, String regex, boolean includeSysTables)	Returns the list of table names that match the specified request
List<TableName>	listTableNamesByNamespace(String name)	Get list of table names by namespace
protected void	login(UserProvider user, String host)	For compatibility, if failed with regionserver credentials, try the master one
static void	main(String[] args)
long	mergeRegions(RegionInfo[] regionsToMerge, boolean forcible, long ng, long nonce)	Merge regions in a table.
long	modifyColumn(TableName tableName, ColumnFamilyDescriptor descriptor, long nonceGroup, long nonce)	Modify the column descriptor of an existing column in an existing table
long	modifyColumnStoreFileTracker(TableName tableName, byte[] family, String dstSFT, long nonceGroup, long nonce)	Modify the store file tracker of an existing column in an existing table
(package private) long	modifyNamespace(NamespaceDescriptor newNsDescriptor, long nonceGroup, long nonce)	Modify an existing Namespace.
long	modifyTable(TableName tableName, TableDescriptor newDescriptor, long nonceGroup, long nonce, boolean reopenRegions)	Modify the descriptor of an existing table
private long	modifyTable(TableName tableName, HMaster.TableDescriptorGetter newDescriptorGetter, long nonceGroup, long nonce, boolean shouldCheckDescriptor)
private long	modifyTable(TableName tableName, HMaster.TableDescriptorGetter newDescriptorGetter, long nonceGroup, long nonce, boolean shouldCheckDescriptor, boolean reopenRegions)
long	modifyTableStoreFileTracker(TableName tableName, String dstSFT, long nonceGroup, long nonce)	Modify the store file tracker of an existing table
void	move(byte[] encodedRegionName, byte[] destServerName)
boolean	normalizeRegions(NormalizeTableFilterParams ntfp, boolean isHighPriority)	Perform normalization of cluster.
void	onConfigurationChange(org.apache.hadoop.conf.Configuration newConf)	This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
private int	putUpJettyServer()
void	recommissionRegionServer(ServerName server, List<byte[]> encodedRegionNames)	Remove decommission marker (previously called 'draining') from a region server to allow regions assignments.
private void	registerConfigurationObservers()
boolean	registerService(org.apache.hbase.thirdparty.com.google.protobuf.Service instance)	Registers a new protocol buffer Service subclass as a master coprocessor endpoint.
void	reloadRegionServerQuotas()
void	remoteProcedureCompleted(long procId, byte[] remoteResultData)
void	remoteProcedureFailed(long procId, RemoteProcedureException error)
long	removeReplicationPeer(String peerId)	Removes a peer and stops the replication
(package private) long	reopenRegions(TableName tableName, List<byte[]> regionNames, long nonceGroup, long nonce)	Reopen regions provided in the argument
boolean	replicationPeerModificationSwitch(boolean on)
void	reportMobCompactionEnd(TableName tableName)
void	reportMobCompactionStart(TableName tableName)
long	restoreSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshotDesc, long nonceGroup, long nonce, boolean restoreAcl, String customSFT)
long	rollAllWALWriters(long nonceGroup, long nonce)	Roll WAL writer for all RegionServers
void	run()
void	runReplicationBarrierCleaner()	Run the ReplicationBarrierChore.
(package private) void	setAssignmentManager(AssignmentManager assignmentManager)
void	setCatalogJanitorChoreForTesting(CatalogJanitor catalogJanitorChore)	Used for testing only to set Mock objects.
void	setCatalogJanitorEnabled(boolean b)	Switch for the background CatalogJanitor thread.
(package private) static void	setDisableBalancerChoreForTest(boolean disable)
void	setHbckChoreForTesting(HbckChore hbckChore)	Used for testing only to set Mock objects.
void	setInitialized(boolean isInitialized)
(package private) void	setLoadBalancer(RSGroupBasedLoadBalancer loadBalancer)
private void	setQuotasObserver(org.apache.hadoop.conf.Configuration conf)
void	setServiceStarted(boolean started)	Mainly used in procedure related tests, where we will restart ProcedureExecutor and AssignmentManager, but we do not want to restart master(to speed up the test), so we need to disable rpc for a while otherwise some critical rpc requests such as reportRegionStateTransition could fail and cause region server to abort.
void	shutdown()	Shutdown the cluster.
boolean	skipRegionManagementAction(String action)	Checks master state before initiating action over region topology.
long	splitRegion(RegionInfo regionInfo, byte[] splitRow, long nonceGroup, long nonce)	Split a region.
private void	startActiveMasterManager(int infoPort)
protected void	startProcedureExecutor()
private void	startServiceThreads()
void	stop(String msg)	Stop this service.
protected void	stopChores()
void	stopMaster()
private void	stopProcedureExecutor()
protected void	stopServiceThreads()
private void	switchSnapshotCleanup(boolean on)
(package private) void	switchSnapshotCleanup(boolean on, boolean synchronous)	Turn on/off Snapshot Cleanup Chore
long	transitReplicationPeerSyncReplicationState(String peerId, SyncReplicationState state)	Set current cluster state for a synchronous replication peer.
long	truncateRegion(RegionInfo regionInfo, long nonceGroup, long nonce)	Truncate region
long	truncateTable(TableName tableName, boolean preserveSplits, long nonceGroup, long nonce)	Truncate a table
private void	tryMigrateMetaLocationsFromZooKeeper()
void	updateConfigurationForQuotasObserver(org.apache.hadoop.conf.Configuration conf)	Adds the MasterQuotasObserver to the list of configured Master observers to automatically remove quotas for a table when that table is deleted.
long	updateReplicationPeerConfig(String peerId, ReplicationPeerConfig peerConfig)	Update the peerConfig for the specified peer
boolean	waitForMetaOnline()	Check hbase:meta is up and ready for reading.
private boolean	waitForNamespaceOnline()	Check hbase:namespace table is assigned.
private void	waitForRegionServers(MonitoredTask status)
private void	warmUpRegion(ServerName server, RegionInfo region)

Methods inherited from class org.apache.hadoop.hbase.HBaseServerBase

closeClusterConnection, closeTableDescriptors, closeZooKeeper, createConnection, getAccessChecker, getAsyncClusterConnection, getChoreService, getConfiguration, getConfigurationManager, getCoordinatedStateManager, getDataRootDir, getEventLoopGroupConfig, getExecutorService, getFileSystem, getInfoServer, getMetaRegionLocationCache, getMsgInterval, getNamedQueueRecorder, getRpcServer, getRpcServices, getStartcode, getTableDescriptors, getWALFileSystem, getWALRootDir, getZKPermissionWatcher, getZooKeeper, initializeFileSystem, initializeMemStoreChunkCreator, installShutdownHook, isAborted, isClusterUp, isShutdownHookInstalled, isStopped, setAbortRequested, setupClusterConnection, shutdownChore, stopChoreService, stopExecutorService, stopInfoServer, toString, updateConfiguration

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, suspend, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.Abortable

abort, isAborted

Methods inherited from interface org.apache.hadoop.hbase.master.MasterServices

getAccessChecker, getExecutorService, getTableDescriptors, getZKPermissionWatcher, isClusterUp, modifyTable

Methods inherited from interface org.apache.hadoop.hbase.Server

createConnection, getAsyncClusterConnection, getAsyncConnection, getChoreService, getConfiguration, getConnection, getCoordinatedStateManager, getFileSystem, getZooKeeper, isStopping

Methods inherited from interface org.apache.hadoop.hbase.Stoppable

isStopped

Field Details

LOG

private static final org.slf4j.Logger LOG

MASTER

public static final String MASTER

See Also:

• Constant Field Values

activeMasterManager

private final ActiveMasterManager activeMasterManager

regionServerTracker

private final RegionServerTracker regionServerTracker

drainingServerTracker

private DrainingServerTracker drainingServerTracker

loadBalancerStateStore

LoadBalancerStateStore loadBalancerStateStore

metaLocationSyncer

private MetaLocationSyncer metaLocationSyncer

masterAddressSyncer

@Private
MasterAddressSyncer masterAddressSyncer

snapshotCleanupStateStore

SnapshotCleanupStateStore snapshotCleanupStateStore

splitOrMergeStateStore

private SplitOrMergeStateStore splitOrMergeStateStore

clusterSchemaService

private ClusterSchemaService clusterSchemaService

HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS

public static final String HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS

See Also:

• Constant Field Values

DEFAULT_HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS

public static final int DEFAULT_HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS

See Also:

• Constant Field Values

HBASE_MASTER_CLEANER_INTERVAL

public static final String HBASE_MASTER_CLEANER_INTERVAL

See Also:

• Constant Field Values

DEFAULT_HBASE_MASTER_CLEANER_INTERVAL

public static final int DEFAULT_HBASE_MASTER_CLEANER_INTERVAL

See Also:

• Constant Field Values

clusterId

private String clusterId

metricsMaster

final MetricsMaster metricsMaster

fileSystemManager

private MasterFileSystem fileSystemManager

walManager

private MasterWalManager walManager

splitWALManager

private SplitWALManager splitWALManager

serverManager

private volatile ServerManager serverManager

assignmentManager

private AssignmentManager assignmentManager

rsGroupInfoManager

private RSGroupInfoManager rsGroupInfoManager

replicationLogCleanerBarrier

private final ReplicationLogCleanerBarrier replicationLogCleanerBarrier

syncReplicationPeerLock

private final Semaphore syncReplicationPeerLock

replicationPeerManager

private ReplicationPeerManager replicationPeerManager

syncReplicationReplayWALManager

private SyncReplicationReplayWALManager syncReplicationReplayWALManager

rsFatals

MemoryBoundedLogMessageBuffer rsFatals

activeMaster

private volatile boolean activeMaster

initialized

private final ProcedureEvent<?> initialized

serviceStarted

volatile boolean serviceStarted

maxBalancingTime

private final int maxBalancingTime

maxRitPercent

private final double maxRitPercent

lockManager

private final LockManager lockManager

balancer

private RSGroupBasedLoadBalancer balancer

balancerChore

private BalancerChore balancerChore

disableBalancerChoreForTest

private static boolean disableBalancerChoreForTest

regionNormalizerManager

private RegionNormalizerManager regionNormalizerManager

clusterStatusChore

private ClusterStatusChore clusterStatusChore

clusterStatusPublisherChore

private ClusterStatusPublisher clusterStatusPublisherChore

snapshotCleanerChore

private SnapshotCleanerChore snapshotCleanerChore

hbckChore

private HbckChore hbckChore

catalogJanitorChore

CatalogJanitor catalogJanitorChore

logCleanerPool

private DirScanPool logCleanerPool

logCleaner

private LogCleaner logCleaner

hfileCleaners

private List<HFileCleaner> hfileCleaners

hfileCleanerPaths

private List<org.apache.hadoop.fs.Path> hfileCleanerPaths

sharedHFileCleanerPool

private DirScanPool sharedHFileCleanerPool

exclusiveHFileCleanerPool

private DirScanPool exclusiveHFileCleanerPool

replicationBarrierCleaner

private ReplicationBarrierCleaner replicationBarrierCleaner

mobFileCleanerChore

private MobFileCleanerChore mobFileCleanerChore

mobFileCompactionChore

private MobFileCompactionChore mobFileCompactionChore

rollingUpgradeChore

private RollingUpgradeChore rollingUpgradeChore

mobCompactionLock

private final IdLock mobCompactionLock

mobCompactionStates

private Map<TableName,AtomicInteger> mobCompactionStates

cpHost

volatile MasterCoprocessorHost cpHost

preLoadTableDescriptors

private final boolean preLoadTableDescriptors

masterActiveTime

private long masterActiveTime

masterFinishedInitializationTime

private long masterFinishedInitializationTime

coprocessorServiceHandlers

Map<String,org.apache.hbase.thirdparty.com.google.protobuf.Service> coprocessorServiceHandlers

snapshotManager

SnapshotManager snapshotManager

mpmHost

private MasterProcedureManagerHost mpmHost

regionsRecoveryChore

private RegionsRecoveryChore regionsRecoveryChore

regionsRecoveryConfigManager

private RegionsRecoveryConfigManager regionsRecoveryConfigManager

quotaManager

private volatile MasterQuotaManager quotaManager

spaceQuotaSnapshotNotifier

private SpaceQuotaSnapshotNotifier spaceQuotaSnapshotNotifier

quotaObserverChore

private QuotaObserverChore quotaObserverChore

snapshotQuotaChore

private SnapshotQuotaObserverChore snapshotQuotaChore

oldWALsDirSizeChore

private OldWALsDirSizeChore oldWALsDirSizeChore

procedureExecutor

private ProcedureExecutor<MasterProcedureEnv> procedureExecutor

procedureStore

private ProcedureStore procedureStore

masterRegion

private MasterRegion masterRegion

rsListStorage

private RegionServerList rsListStorage

tableStateManager

private TableStateManager tableStateManager

masterJettyServer

private org.apache.hbase.thirdparty.org.eclipse.jetty.server.Server masterJettyServer

jetty server for master to redirect requests to regionserver infoServer

maintenanceMode

private final boolean maintenanceMode

MAINTENANCE_MODE

static final String MAINTENANCE_MODE

See Also:

• Constant Field Values

maintenanceRegionServer

private JVMClusterUtil.RegionServerThread maintenanceRegionServer

cachedClusterId

private final CachedClusterId cachedClusterId

WARMUP_BEFORE_MOVE

public static final String WARMUP_BEFORE_MOVE

See Also:

• Constant Field Values

DEFAULT_WARMUP_BEFORE_MOVE

private static final boolean DEFAULT_WARMUP_BEFORE_MOVE

See Also:

• Constant Field Values

HBASE_MASTER_RSPROC_DISPATCHER_CLASS

public static final String HBASE_MASTER_RSPROC_DISPATCHER_CLASS

Use RSProcedureDispatcher instance to initiate master -> rs remote procedure execution. Use this config to extend RSProcedureDispatcher (mainly for testing purpose).

See Also:

• Constant Field Values

DEFAULT_HBASE_MASTER_RSPROC_DISPATCHER_CLASS

private static final String DEFAULT_HBASE_MASTER_RSPROC_DISPATCHER_CLASS

startupTaskGroup

private TaskGroup startupTaskGroup

replicationPeerModificationStateStore

private ReplicationPeerModificationStateStore replicationPeerModificationStateStore

Store whether we allow replication peer modification operations.

Constructor Details

HMaster

public HMaster(org.apache.hadoop.conf.Configuration conf)
        throws IOException

Initializes the HMaster. The steps are as follows:

1. Initialize the local HRegionServer
2. Start the ActiveMasterManager.

Remaining steps of initialization occur in finishActiveMasterInitialization() after the master becomes the active one.

Throws:

IOException

Method Details

createActiveMasterManager

protected ActiveMasterManager createActiveMasterManager(ZKWatcher zk,
 ServerName sn,
 Server server)
                                                 throws InterruptedIOException

Protected to have custom implementations in tests override the default ActiveMaster implementation.

Throws:

InterruptedIOException

getUseThisHostnameInstead

protected String getUseThisHostnameInstead(org.apache.hadoop.conf.Configuration conf)

Specified by:

getUseThisHostnameInstead in class HBaseServerBase<MasterRpcServices>

registerConfigurationObservers

private void registerConfigurationObservers()

run

public void run()

Specified by:

run in interface Runnable

Overrides:

run in class Thread

putUpJettyServer

private int putUpJettyServer()
                      throws IOException

Throws:

IOException

login

protected void login(UserProvider user,
 String host)
              throws IOException

For compatibility, if failed with regionserver credentials, try the master one

Specified by:

login in class HBaseServerBase<MasterRpcServices>

Throws:

IOException

getMasterRpcServices

public MasterRpcServices getMasterRpcServices()

getCoprocessorHost

protected MasterCoprocessorHost getCoprocessorHost()

Specified by:

getCoprocessorHost in class HBaseServerBase<MasterRpcServices>

balanceSwitch

public boolean balanceSwitch(boolean b)
                      throws IOException

Throws:

IOException

getProcessName

protected String getProcessName()

Specified by:

getProcessName in class HBaseServerBase<MasterRpcServices>

canCreateBaseZNode

protected boolean canCreateBaseZNode()

Specified by:

canCreateBaseZNode in class HBaseServerBase<MasterRpcServices>

canUpdateTableDescriptor

protected boolean canUpdateTableDescriptor()

Specified by:

canUpdateTableDescriptor in class HBaseServerBase<MasterRpcServices>

cacheTableDescriptor

protected boolean cacheTableDescriptor()

Specified by:

cacheTableDescriptor in class HBaseServerBase<MasterRpcServices>

createRpcServices

protected MasterRpcServices createRpcServices()
                                       throws IOException

Specified by:

createRpcServices in class HBaseServerBase<MasterRpcServices>

Throws:

IOException

configureInfoServer

protected void configureInfoServer(InfoServer infoServer)

Specified by:

configureInfoServer in class HBaseServerBase<MasterRpcServices>

buildApiV1Servlet

private org.apache.hbase.thirdparty.org.eclipse.jetty.ee8.servlet.ServletHolder buildApiV1Servlet()

buildHbckServlet

private org.apache.hbase.thirdparty.org.eclipse.jetty.ee8.servlet.ServletHolder buildHbckServlet()

getDumpServlet

protected Class<? extends javax.servlet.http.HttpServlet> getDumpServlet()

Specified by:

getDumpServlet in class HBaseServerBase<MasterRpcServices>

getMasterMetrics

public MetricsMaster getMasterMetrics()

Description copied from interface: MasterServices

Returns Master's instance of MetricsMaster

Specified by:

getMasterMetrics in interface MasterServices

initializeZKBasedSystemTrackers

private void initializeZKBasedSystemTrackers()
                                      throws IOException,
org.apache.zookeeper.KeeperException,
ReplicationException,
DeserializationException

Initialize all ZK based system trackers. But do not include RegionServerTracker, it should have already been initialized along with ServerManager.

Throws:

IOException

org.apache.zookeeper.KeeperException

ReplicationException

DeserializationException

createAssignmentManager

@Private
protected AssignmentManager createAssignmentManager(MasterServices master,
 MasterRegion masterRegion)

tryMigrateMetaLocationsFromZooKeeper

private void tryMigrateMetaLocationsFromZooKeeper()
                                           throws IOException,
org.apache.zookeeper.KeeperException

Throws:

IOException

org.apache.zookeeper.KeeperException

finishActiveMasterInitialization

private void finishActiveMasterInitialization()
                                       throws IOException,
InterruptedException,
org.apache.zookeeper.KeeperException,
ReplicationException,
DeserializationException

Finish initialization of HMaster after becoming the primary master.

The startup order is a bit complicated but very important, do not change it unless you know what you are doing.

1. Initialize file system based components - file system manager, wal manager, table descriptors, etc
2. Publish cluster id
3. Here comes the most complicated part - initialize server manager, assignment manager and region server tracker
   1. Create server manager
   2. Create master local region
   3. Create procedure executor, load the procedures, but do not start workers. We will start it later after we finish scheduling SCPs to avoid scheduling duplicated SCPs for the same server
   4. Create assignment manager and start it, load the meta region state, but do not load data from meta region
   5. Start region server tracker, construct the online servers set and find out dead servers and schedule SCP for them. The online servers will be constructed by scanning zk, and we will also scan the wal directory and load from master local region to find out possible live region servers, and the differences between these two sets are the dead servers
4. If this is a new deploy, schedule a InitMetaProcedure to initialize meta
5. Start necessary service threads - balancer, catalog janitor, executor services, and also the procedure executor, etc. Notice that the balancer must be created first as assignment manager may use it when assigning regions.
6. Wait for meta to be initialized if necessary, start table state manager.
7. Wait for enough region servers to check-in
8. Let assignment manager load data from meta and construct region states
9. Start all other things such as chore services, etc

Notice that now we will not schedule a special procedure to make meta online(unless the first time where meta has not been created yet), we will rely on SCP to bring meta online.

Throws:

IOException

InterruptedException

org.apache.zookeeper.KeeperException

ReplicationException

DeserializationException

setHbckChoreForTesting

public void setHbckChoreForTesting(HbckChore hbckChore)

Used for testing only to set Mock objects.

Parameters:

hbckChore - hbckChore

setCatalogJanitorChoreForTesting

public void setCatalogJanitorChoreForTesting(CatalogJanitor catalogJanitorChore)

Used for testing only to set Mock objects.

Parameters:

catalogJanitorChore - catalogJanitorChore

createMissingCFsInMetaDuringUpgrade

private void createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
                                          throws IOException

Throws:

IOException

waitForMetaOnline

@Private
public boolean waitForMetaOnline()

Check hbase:meta is up and ready for reading. For use during Master startup only.

Returns:

True if meta is UP and online and startup can progress. Otherwise, meta is not online and we will hold here until operator intervention.

isRegionOnline

private boolean isRegionOnline(RegionInfo ri)

Returns:

True if region is online and scannable else false if an error or shutdown (Otherwise we just block in here holding up all forward-progess).

waitForNamespaceOnline

private boolean waitForNamespaceOnline()
                                throws IOException

Check hbase:namespace table is assigned. If not, startup will hang looking for the ns table

This is for rolling upgrading, later we will migrate the data in ns table to the ns family of meta table. And if this is a new cluster, this method will return immediately as there will be no namespace table/region.

Returns:

True if namespace table is up/online.

Throws:

IOException

updateConfigurationForQuotasObserver

@Private
public void updateConfigurationForQuotasObserver(org.apache.hadoop.conf.Configuration conf)

Adds the MasterQuotasObserver to the list of configured Master observers to automatically remove quotas for a table when that table is deleted.

initMobCleaner

private void initMobCleaner()

createServerManager

@Private
protected ServerManager createServerManager(MasterServices master,
 RegionServerList storage)
                                     throws IOException

Create a ServerManager instance.

Will be overridden in tests.

Throws:

IOException

waitForRegionServers

private void waitForRegionServers(MonitoredTask status)
                           throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

initClusterSchemaService

@Private
protected void initClusterSchemaService()
                                 throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

initQuotaManager

private void initQuotaManager()
                       throws IOException

Throws:

IOException

createQuotaSnapshotNotifier

private SpaceQuotaSnapshotNotifier createQuotaSnapshotNotifier()

isCatalogJanitorEnabled

public boolean isCatalogJanitorEnabled()

isCleanerChoreEnabled

boolean isCleanerChoreEnabled()

getServerManager

public ServerManager getServerManager()

Description copied from interface: MasterServices

Returns Master's ServerManager instance.

Specified by:

getServerManager in interface MasterServices

getMasterFileSystem

public MasterFileSystem getMasterFileSystem()

Description copied from interface: MasterServices

Returns Master's filesystem MasterFileSystem utility class.

Specified by:

getMasterFileSystem in interface MasterServices

getMasterWalManager

public MasterWalManager getMasterWalManager()

Description copied from interface: MasterServices

Returns Master's WALs MasterWalManager utility class.

Specified by:

getMasterWalManager in interface MasterServices

getSplitWALManager

public SplitWALManager getSplitWALManager()

Description copied from interface: MasterServices

Returns return null if current is zk-based WAL splitting

Specified by:

getSplitWALManager in interface MasterServices

getTableStateManager

public TableStateManager getTableStateManager()

Description copied from interface: MasterServices

Returns Master's instance of TableStateManager

Specified by:

getTableStateManager in interface MasterServices

startServiceThreads

private void startServiceThreads()
                          throws IOException

Throws:

IOException

stopServiceThreads

protected void stopServiceThreads()

createProcedureExecutor

private void createProcedureExecutor()
                              throws IOException

Throws:

IOException

startProcedureExecutor

protected void startProcedureExecutor()
                               throws IOException

Throws:

IOException

switchSnapshotCleanup

void switchSnapshotCleanup(boolean on,
 boolean synchronous)
                    throws IOException

Turn on/off Snapshot Cleanup Chore

Parameters:

on - indicates whether Snapshot Cleanup Chore is to be run

Throws:

IOException

switchSnapshotCleanup

private void switchSnapshotCleanup(boolean on)
                            throws IOException

Throws:

IOException

stopProcedureExecutor

private void stopProcedureExecutor()

stopChores

protected void stopChores()

Specified by:

stopChores in class HBaseServerBase<MasterRpcServices>

getRemoteInetAddress

InetAddress getRemoteInetAddress(int port,
 long serverStartCode)
                          throws UnknownHostException

Returns Get remote side's InetAddress

Throws:

UnknownHostException

getMaxBalancingTime

private int getMaxBalancingTime()

Returns Maximum time we should run balancer for

getMaxRegionsInTransition

private int getMaxRegionsInTransition()

Returns Maximum number of regions in transition

balanceThrottling

private void balanceThrottling(long nextBalanceStartTime,
 int maxRegionsInTransition,
 long cutoffTime)

It first sleep to the next balance plan start time. Meanwhile, throttling by the max number regions in transition to protect availability.

Parameters:

nextBalanceStartTime - The next balance plan start time

maxRegionsInTransition - max number of regions in transition

cutoffTime - when to exit balancer

balance

public BalanceResponse balance()
                        throws IOException

Throws:

IOException

balanceOrUpdateMetrics

public BalanceResponse balanceOrUpdateMetrics()
                                       throws IOException

Trigger a normal balance, see balance() . If the balance is not executed this time, the metrics related to the balance will be updated. When balance is running, related metrics will be updated at the same time. But if some checking logic failed and cause the balancer exit early, we lost the chance to update balancer metrics. This will lead to user missing the latest balancer info.

Throws:

IOException

skipRegionManagementAction

public boolean skipRegionManagementAction(String action)

Checks master state before initiating action over region topology.

Specified by:

skipRegionManagementAction in interface MasterServices

Parameters:

action - the name of the action under consideration, for logging.

Returns:

true when the caller should exit early, false otherwise.

balance

public BalanceResponse balance(BalanceRequest request)
                        throws IOException

Throws:

IOException

executeRegionPlansWithThrottling

public List<RegionPlan> executeRegionPlansWithThrottling(List<RegionPlan> plans)

Execute region plans with throttling

Specified by:

executeRegionPlansWithThrottling in interface MasterServices

Parameters:

plans - to execute

Returns:

succeeded plans

getRegionNormalizerManager

public RegionNormalizerManager getRegionNormalizerManager()

Description copied from interface: MasterServices

Returns Master's instance of RegionNormalizerManager

Specified by:

getRegionNormalizerManager in interface MasterServices

normalizeRegions

public boolean normalizeRegions(NormalizeTableFilterParams ntfp,
 boolean isHighPriority)
                         throws IOException

Description copied from interface: MasterServices

Perform normalization of cluster.

Specified by:

normalizeRegions in interface MasterServices

Parameters:

ntfp - Selection criteria for identifying which tables to normalize.

isHighPriority - true when these requested tables should skip to the front of the queue.

Returns:

true when the request was submitted, false otherwise.

Throws:

IOException

getClientIdAuditPrefix

public String getClientIdAuditPrefix()

Returns Client info for use as prefix on an audit log string; who did an action

Specified by:

getClientIdAuditPrefix in interface MasterServices

setCatalogJanitorEnabled

public void setCatalogJanitorEnabled(boolean b)

Switch for the background CatalogJanitor thread. Used for testing. The thread will continue to run. It will just be a noop if disabled.

Parameters:

b - If false, the catalog janitor won't do anything.

mergeRegions

public long mergeRegions(RegionInfo[] regionsToMerge,
 boolean forcible,
 long ng,
 long nonce)
                  throws IOException

Description copied from interface: MasterServices

Merge regions in a table.

Specified by:

mergeRegions in interface MasterServices

Parameters:

regionsToMerge - daughter regions to merge

forcible - whether to force to merge even two regions are not adjacent

ng - used to detect duplicate

nonce - used to detect duplicate

Returns:

procedure Id

Throws:

IOException

splitRegion

public long splitRegion(RegionInfo regionInfo,
 byte[] splitRow,
 long nonceGroup,
 long nonce)
                 throws IOException

Description copied from interface: MasterServices

Split a region.

Specified by:

splitRegion in interface MasterServices

Parameters:

regionInfo - region to split

splitRow - split point

nonceGroup - used to detect duplicate

nonce - used to detect duplicate

Returns:

procedure Id

Throws:

IOException

warmUpRegion

private void warmUpRegion(ServerName server,
 RegionInfo region)

move

@Private
public void move(byte[] encodedRegionName,
 byte[] destServerName)
          throws IOException

Throws:

IOException

createTable

public long createTable(TableDescriptor tableDescriptor,
 byte[][] splitKeys,
 long nonceGroup,
 long nonce)
                 throws IOException

Description copied from interface: MasterServices

Create a table using the given table definition.

Specified by:

createTable in interface MasterServices

Parameters:

tableDescriptor - The table definition

splitKeys - Starting row keys for the initial table regions. If null a single region is created.

Throws:

IOException

createSystemTable

public long createSystemTable(TableDescriptor tableDescriptor)
                       throws IOException

Description copied from interface: MasterServices

Create a system table using the given table definition.

Specified by:

createSystemTable in interface MasterServices

Parameters:

tableDescriptor - The system table definition a single region is created.

Throws:

IOException

startActiveMasterManager

private void startActiveMasterManager(int infoPort)
                               throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

isCatalogTable

private static boolean isCatalogTable(TableName tableName)

deleteTable

public long deleteTable(TableName tableName,
 long nonceGroup,
 long nonce)
                 throws IOException

Description copied from interface: MasterServices

Delete a table

Specified by:

deleteTable in interface MasterServices

Parameters:

tableName - The table name

Throws:

IOException

truncateTable

public long truncateTable(TableName tableName,
 boolean preserveSplits,
 long nonceGroup,
 long nonce)
                   throws IOException

Description copied from interface: MasterServices

Truncate a table

Specified by:

truncateTable in interface MasterServices

Parameters:

tableName - The table name

preserveSplits - True if the splits should be preserved

Throws:

IOException

truncateRegion

public long truncateRegion(RegionInfo regionInfo,
 long nonceGroup,
 long nonce)
                    throws IOException

Description copied from interface: MasterServices

Truncate region

Specified by:

truncateRegion in interface MasterServices

Parameters:

regionInfo - region to be truncated

nonceGroup - the nonce group

nonce - the nonce

Returns:

procedure Id

Throws:

IOException

addColumn

public long addColumn(TableName tableName,
 ColumnFamilyDescriptor column,
 long nonceGroup,
 long nonce)
               throws IOException

Description copied from interface: MasterServices

Add a new column to an existing table

Specified by:

addColumn in interface MasterServices

Parameters:

tableName - The table name

column - The column definition

Throws:

IOException

modifyColumn

public long modifyColumn(TableName tableName,
 ColumnFamilyDescriptor descriptor,
 long nonceGroup,
 long nonce)
                  throws IOException

Description copied from interface: MasterServices

Modify the column descriptor of an existing column in an existing table

Specified by:

modifyColumn in interface MasterServices

Parameters:

tableName - The table name

descriptor - The updated column definition

Throws:

IOException

modifyColumnStoreFileTracker

public long modifyColumnStoreFileTracker(TableName tableName,
 byte[] family,
 String dstSFT,
 long nonceGroup,
 long nonce)
                                  throws IOException

Description copied from interface: MasterServices

Modify the store file tracker of an existing column in an existing table

Specified by:

modifyColumnStoreFileTracker in interface MasterServices

Throws:

IOException

deleteColumn

public long deleteColumn(TableName tableName,
 byte[] columnName,
 long nonceGroup,
 long nonce)
                  throws IOException

Description copied from interface: MasterServices

Delete a column from an existing table

Specified by:

deleteColumn in interface MasterServices

Parameters:

tableName - The table name

columnName - The column name

Throws:

IOException

enableTable

public long enableTable(TableName tableName,
 long nonceGroup,
 long nonce)
                 throws IOException

Description copied from interface: MasterServices

Enable an existing table

Specified by:

enableTable in interface MasterServices

Parameters:

tableName - The table name

Throws:

IOException

disableTable

public long disableTable(TableName tableName,
 long nonceGroup,
 long nonce)
                  throws IOException

Description copied from interface: MasterServices

Disable an existing table

Specified by:

disableTable in interface MasterServices

Parameters:

tableName - The table name

Throws:

IOException

modifyTable

private long modifyTable(TableName tableName,
 HMaster.TableDescriptorGetter newDescriptorGetter,
 long nonceGroup,
 long nonce,
 boolean shouldCheckDescriptor)
                  throws IOException

Throws:

IOException

modifyTable

private long modifyTable(TableName tableName,
 HMaster.TableDescriptorGetter newDescriptorGetter,
 long nonceGroup,
 long nonce,
 boolean shouldCheckDescriptor,
 boolean reopenRegions)
                  throws IOException

Throws:

IOException

modifyTable

public long modifyTable(TableName tableName,
 TableDescriptor newDescriptor,
 long nonceGroup,
 long nonce,
 boolean reopenRegions)
                 throws IOException

Description copied from interface: MasterServices

Modify the descriptor of an existing table

Specified by:

modifyTable in interface MasterServices

Parameters:

tableName - The table name

newDescriptor - The updated table descriptor

reopenRegions - Whether to reopen regions after modifying the table descriptor

Throws:

IOException

modifyTableStoreFileTracker

public long modifyTableStoreFileTracker(TableName tableName,
 String dstSFT,
 long nonceGroup,
 long nonce)
                                 throws IOException

Description copied from interface: MasterServices

Modify the store file tracker of an existing table

Specified by:

modifyTableStoreFileTracker in interface MasterServices

Throws:

IOException

restoreSnapshot

public long restoreSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshotDesc,
 long nonceGroup,
 long nonce,
 boolean restoreAcl,
 String customSFT)
                     throws IOException

Throws:

IOException

checkTableExists

private void checkTableExists(TableName tableName)
                       throws IOException,
TableNotFoundException

Throws:

IOException

TableNotFoundException

checkTableModifiable

public void checkTableModifiable(TableName tableName)
                          throws IOException,
TableNotFoundException,
TableNotDisabledException

Description copied from interface: MasterServices

Check table is modifiable; i.e. exists and is offline.

Specified by:

checkTableModifiable in interface MasterServices

Parameters:

tableName - Name of table to check.

Throws:

IOException

TableNotFoundException

TableNotDisabledException

reloadRegionServerQuotas

public void reloadRegionServerQuotas()

getClusterMetricsWithoutCoprocessor

public ClusterMetrics getClusterMetricsWithoutCoprocessor()
                                                   throws InterruptedIOException

Throws:

InterruptedIOException

getClusterMetricsWithoutCoprocessor

public ClusterMetrics getClusterMetricsWithoutCoprocessor(EnumSet<ClusterMetrics.Option> options)
                                                   throws InterruptedIOException

Throws:

InterruptedIOException

getUnknownServers

private List<ServerName> getUnknownServers()

getOnlineServers

private Map<ServerName,ServerMetrics> getOnlineServers()

getClusterMetrics

public ClusterMetrics getClusterMetrics()
                                 throws IOException

Returns cluster status

Throws:

IOException

getClusterMetrics

public ClusterMetrics getClusterMetrics(EnumSet<ClusterMetrics.Option> options)
                                 throws IOException

Throws:

IOException

getActiveMasterInfoPort

public int getActiveMasterInfoPort()

Returns info port of active master or 0 if any exception occurs.

getBackupMasterInfoPort

public int getBackupMasterInfoPort(ServerName sn)

Parameters:

sn - is ServerName of the backup master

Returns:

info port of backup master or 0 if any exception occurs.

getLoadedCoprocessors

public static String getLoadedCoprocessors()

The set of loaded coprocessors is stored in a static set. Since it's statically allocated, it does not require that HMaster's cpHost be initialized prior to accessing it.

Returns:

a String representation of the set of names of the loaded coprocessors.

getMasterStartTime

public long getMasterStartTime()

Returns timestamp in millis when HMaster was started.

getMasterActiveTime

public long getMasterActiveTime()

Returns timestamp in millis when HMaster became the active master.

Specified by:

getMasterActiveTime in interface MasterServices

getMasterFinishedInitializationTime

public long getMasterFinishedInitializationTime()

Returns timestamp in millis when HMaster finished becoming the active master

getNumWALFiles

public int getNumWALFiles()

getProcedureStore

public ProcedureStore getProcedureStore()

getRegionServerInfoPort

public int getRegionServerInfoPort(ServerName sn)

getRegionServerVersion

public String getRegionServerVersion(ServerName sn)

Specified by:

getRegionServerVersion in interface MasterServices

checkIfShouldMoveSystemRegionAsync

public void checkIfShouldMoveSystemRegionAsync()

Description copied from interface: MasterServices

Called when a new RegionServer is added to the cluster. Checks if new server has a newer version than any existing server and will move system tables there if so.

Specified by:

checkIfShouldMoveSystemRegionAsync in interface MasterServices

getMasterCoprocessors

public String[] getMasterCoprocessors()

Returns array of coprocessor SimpleNames.

abort

public void abort(String reason,
 Throwable cause)

Description copied from interface: Abortable

Abort the server or client.

Specified by:

abort in interface Abortable

Parameters:

reason - Why we're aborting.

cause - Throwable that caused abort. Can be null.

getMasterCoprocessorHost

public MasterCoprocessorHost getMasterCoprocessorHost()

Description copied from interface: MasterServices

Returns Master's instance of MasterCoprocessorHost

Specified by:

getMasterCoprocessorHost in interface MasterServices

getMasterQuotaManager

public MasterQuotaManager getMasterQuotaManager()

Description copied from interface: MasterServices

Returns Master's instance of MasterQuotaManager

Specified by:

getMasterQuotaManager in interface MasterServices

getMasterProcedureExecutor

public ProcedureExecutor<MasterProcedureEnv> getMasterProcedureExecutor()

Description copied from interface: MasterServices

Returns Master's instance of ProcedureExecutor

Specified by:

getMasterProcedureExecutor in interface MasterServices

getServerName

public ServerName getServerName()

Description copied from interface: Server

Returns The unique server name for this server.

Specified by:

getServerName in interface Server

Overrides:

getServerName in class HBaseServerBase<MasterRpcServices>

getAssignmentManager

public AssignmentManager getAssignmentManager()

Description copied from interface: MasterServices

Returns Master's instance of the AssignmentManager

Specified by:

getAssignmentManager in interface MasterServices

getCatalogJanitor

public CatalogJanitor getCatalogJanitor()

Description copied from interface: MasterServices

Returns Master's instance of CatalogJanitor

Specified by:

getCatalogJanitor in interface MasterServices

getRegionServerFatalLogBuffer

public MemoryBoundedLogMessageBuffer getRegionServerFatalLogBuffer()

getStartupProgress

public TaskGroup getStartupProgress()

shutdown

public void shutdown()
              throws IOException

Shutdown the cluster. Master runs a coordinated stop of all RegionServers and then itself.

Throws:

IOException

stopMaster

public void stopMaster()
                throws IOException

Throws:

IOException

stop

public void stop(String msg)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Specified by:

stop in interface Stoppable

Parameters:

msg - Why we're stopping.

checkServiceStarted

protected void checkServiceStarted()
                            throws ServerNotRunningYetException

Throws:

ServerNotRunningYetException

checkInitialized

void checkInitialized()
               throws PleaseHoldException,
ServerNotRunningYetException,
MasterNotRunningException,
MasterStoppedException

Throws:

PleaseHoldException

ServerNotRunningYetException

MasterNotRunningException

MasterStoppedException

isActiveMaster

public boolean isActiveMaster()

Report whether this master is currently the active master or not. If not active master, we are parked on ZK waiting to become active. This method is used for testing.

Specified by:

isActiveMaster in interface MasterServices

Returns:

true if active master, false if not.

isInitialized

public boolean isInitialized()

Report whether this master has completed with its initialization and is ready. If ready, the master is also the active master. A standby master is never ready. This method is used for testing.

Specified by:

isInitialized in interface MasterServices

Returns:

true if master is ready to go, false if not.

isOnline

public boolean isOnline()

Report whether this master is started This method is used for testing.

Returns:

true if master is ready to go, false if not.

isInMaintenanceMode

public boolean isInMaintenanceMode()

Report whether this master is in maintenance mode.

Specified by:

isInMaintenanceMode in interface MasterServices

Returns:

true if master is in maintenanceMode

setInitialized

public void setInitialized(boolean isInitialized)

setServiceStarted

public void setServiceStarted(boolean started)

Mainly used in procedure related tests, where we will restart ProcedureExecutor and AssignmentManager, but we do not want to restart master(to speed up the test), so we need to disable rpc for a while otherwise some critical rpc requests such as reportRegionStateTransition could fail and cause region server to abort.

getInitializedEvent

public ProcedureEvent<?> getInitializedEvent()

Description copied from interface: MasterServices

Returns Tripped when Master has finished initialization.

Specified by:

getInitializedEvent in interface MasterServices

getAverageLoad

public double getAverageLoad()

Compute the average load across all region servers. Currently, this uses a very naive computation - just uses the number of regions being served, ignoring stats about number of requests.

Returns:

the average load

registerService

public boolean registerService(org.apache.hbase.thirdparty.com.google.protobuf.Service instance)

Description copied from interface: MasterServices

Registers a new protocol buffer Service subclass as a master coprocessor endpoint.

Only a single instance may be registered for a given Service subclass (the instances are keyed on Descriptors.ServiceDescriptor.getFullName(). After the first registration, subsequent calls with the same service name will fail with a return value of false.

Specified by:

registerService in interface MasterServices

Parameters:

instance - the Service subclass instance to expose as a coprocessor endpoint

Returns:

true if the registration was successful, false otherwise

constructMaster

public static HMaster constructMaster(Class<? extends HMaster> masterClass,
 org.apache.hadoop.conf.Configuration conf)

Utility for constructing an instance of the passed HMaster class.

Returns:

HMaster instance.

main

public static void main(String[] args)

See Also:

• HMasterCommandLine

getHFileCleaner

public HFileCleaner getHFileCleaner()

getHFileCleaners

public List<HFileCleaner> getHFileCleaners()

getLogCleaner

public LogCleaner getLogCleaner()

getSnapshotManager

public SnapshotManager getSnapshotManager()

Returns the underlying snapshot manager

Specified by:

getSnapshotManager in interface MasterServices

getMasterProcedureManagerHost

public MasterProcedureManagerHost getMasterProcedureManagerHost()

Returns the underlying MasterProcedureManagerHost

Specified by:

getMasterProcedureManagerHost in interface MasterServices

getClusterSchema

public ClusterSchema getClusterSchema()

Description copied from interface: MasterServices

Returns Master's instance of ClusterSchema

Specified by:

getClusterSchema in interface MasterServices

createNamespace

long createNamespace(NamespaceDescriptor namespaceDescriptor,
 long nonceGroup,
 long nonce)
              throws IOException

Create a new Namespace.

Parameters:

namespaceDescriptor - descriptor for new Namespace

nonceGroup - Identifier for the source of the request, a client or process.

nonce - A unique identifier for this operation from the client or process identified by nonceGroup (the source must ensure each operation gets a unique id).

Returns:

procedure id

Throws:

IOException

modifyNamespace

long modifyNamespace(NamespaceDescriptor newNsDescriptor,
 long nonceGroup,
 long nonce)
              throws IOException

Modify an existing Namespace.

Parameters:

nonceGroup - Identifier for the source of the request, a client or process.

nonce - A unique identifier for this operation from the client or process identified by nonceGroup (the source must ensure each operation gets a unique id).

Returns:

procedure id

Throws:

IOException

deleteNamespace

long deleteNamespace(String name,
 long nonceGroup,
 long nonce)
              throws IOException

Delete an existing Namespace. Only empty Namespaces (no tables) can be removed.

Parameters:

nonceGroup - Identifier for the source of the request, a client or process.

nonce - A unique identifier for this operation from the client or process identified by nonceGroup (the source must ensure each operation gets a unique id).

Returns:

procedure id

Throws:

IOException

getNamespace

NamespaceDescriptor getNamespace(String name)
                          throws IOException

Get a Namespace

Parameters:

name - Name of the Namespace

Returns:

Namespace descriptor for name

Throws:

IOException

getNamespaces

List<NamespaceDescriptor> getNamespaces()
                                 throws IOException

Get all Namespaces

Returns:

All Namespace descriptors

Throws:

IOException

listNamespaces

public List<String> listNamespaces()
                            throws IOException

List namespace names

Returns:

All namespace names

Throws:

IOException

listTableNamesByNamespace

public List<TableName> listTableNamesByNamespace(String name)
                                          throws IOException

Description copied from interface: MasterServices

Get list of table names by namespace

Specified by:

listTableNamesByNamespace in interface MasterServices

Parameters:

name - namespace name

Returns:

table names

Throws:

IOException

listTableDescriptorsByNamespace

public List<TableDescriptor> listTableDescriptorsByNamespace(String name)
                                                      throws IOException

Description copied from interface: MasterServices

Get list of table descriptors by namespace

Specified by:

listTableDescriptorsByNamespace in interface MasterServices

Parameters:

name - namespace name

Throws:

IOException

abortProcedure

public boolean abortProcedure(long procId,
 boolean mayInterruptIfRunning)
                       throws IOException

Description copied from interface: MasterServices

Abort a procedure.

Specified by:

abortProcedure in interface MasterServices

Parameters:

procId - ID of the procedure

mayInterruptIfRunning - if the proc completed at least one step, should it be aborted?

Returns:

true if aborted, false if procedure already completed or does not exist

Throws:

IOException

getProcedures

public List<Procedure<?>> getProcedures()
                                 throws IOException

Description copied from interface: MasterServices

Get procedures

Specified by:

getProcedures in interface MasterServices

Returns:

procedure list

Throws:

IOException

getLocks

public List<LockedResource> getLocks()
                              throws IOException

Description copied from interface: MasterServices

Get locks

Specified by:

getLocks in interface MasterServices

Returns:

lock list

Throws:

IOException

listTableDescriptors

public List<TableDescriptor> listTableDescriptors(String namespace,
 String regex,
 List<TableName> tableNameList,
 boolean includeSysTables)
                                           throws IOException

Returns the list of table descriptors that match the specified request

Parameters:

namespace - the namespace to query, or null if querying for all

regex - The regular expression to match against, or null if querying for all

tableNameList - the list of table names, or null if querying for all

includeSysTables - False to match only against userspace tables

Returns:

the list of table descriptors

Throws:

IOException

listTableNames

public List<TableName> listTableNames(String namespace,
 String regex,
 boolean includeSysTables)
                               throws IOException

Returns the list of table names that match the specified request

Parameters:

regex - The regular expression to match against, or null if querying for all

namespace - the namespace to query, or null if querying for all

includeSysTables - False to match only against userspace tables

Returns:

the list of table names

Throws:

IOException

getTableDescriptors

private List<TableDescriptor> getTableDescriptors(List<TableDescriptor> htds,
 String namespace,
 String regex,
 List<TableName> tableNameList,
 boolean includeSysTables)
                                           throws IOException

Return a list of table table descriptors after applying any provided filter parameters. Note that the user-facing description of this filter logic is presented on the class-level javadoc of NormalizeTableFilterParams.

Throws:

IOException

filterTablesByRegex

private static void filterTablesByRegex(Collection<TableDescriptor> descriptors,
 Pattern pattern)

Removes the table descriptors that don't match the pattern.

Parameters:

descriptors - list of table descriptors to filter

pattern - the regex to use

getLastMajorCompactionTimestamp

public long getLastMajorCompactionTimestamp(TableName table)
                                     throws IOException

Specified by:

getLastMajorCompactionTimestamp in interface MasterServices

Parameters:

table - the table for which last successful major compaction time is queried

Returns:

the timestamp of the last successful major compaction for the passed table, or 0 if no HFile resulting from a major compaction exists

Throws:

IOException

getLastMajorCompactionTimestampForRegion

public long getLastMajorCompactionTimestampForRegion(byte[] regionName)
                                              throws IOException

Description copied from interface: MasterServices

Returns the timestamp of the last successful major compaction for the passed region or 0 if no HFile resulting from a major compaction exists

Specified by:

getLastMajorCompactionTimestampForRegion in interface MasterServices

Throws:

IOException

getMobCompactionState

public org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.GetRegionInfoResponse.CompactionState getMobCompactionState(TableName tableName)

Gets the mob file compaction state for a specific table. Whether all the mob files are selected is known during the compaction execution, but the statistic is done just before compaction starts, it is hard to know the compaction type at that time, so the rough statistics are chosen for the mob file compaction. Only two compaction states are available, CompactionState.MAJOR_AND_MINOR and CompactionState.NONE.

Parameters:

tableName - The current table name.

Returns:

If a given table is in mob file compaction now.

reportMobCompactionStart

public void reportMobCompactionStart(TableName tableName)
                              throws IOException

Throws:

IOException

reportMobCompactionEnd

public void reportMobCompactionEnd(TableName tableName)
                            throws IOException

Throws:

IOException

isBalancerOn

public boolean isBalancerOn()

Queries the state of the LoadBalancerStateStore. If the balancer is not initialized, false is returned.

Specified by:

isBalancerOn in interface MasterServices

Returns:

The state of the load balancer, or false if the load balancer isn't defined.

isNormalizerOn

public boolean isNormalizerOn()

Queries the state of the RegionNormalizerStateStore. If it's not initialized, false is returned.

isSplitOrMergeEnabled

public boolean isSplitOrMergeEnabled(MasterSwitchType switchType)

Queries the state of the SplitOrMergeStateStore. If it is not initialized, false is returned. If switchType is illegal, false will return.

Specified by:

isSplitOrMergeEnabled in interface MasterServices

Parameters:

switchType - see MasterSwitchType

Returns:

The state of the switch

getLoadBalancerClassName

public String getLoadBalancerClassName()

Fetch the configured LoadBalancer class name. If none is set, a default is returned.

Notice that, the base load balancer will always be RSGroupBasedLoadBalancer now, so this method will return the balancer used inside each rs group.

Returns:

The name of the LoadBalancer in use.

getSplitOrMergeStateStore

public SplitOrMergeStateStore getSplitOrMergeStateStore()

getLoadBalancer

public RSGroupBasedLoadBalancer getLoadBalancer()

Description copied from interface: MasterServices

Returns load balancer

Specified by:

getLoadBalancer in interface MasterServices

getFavoredNodesManager

public FavoredNodesManager getFavoredNodesManager()

Description copied from interface: MasterServices

Returns Favored Nodes Manager

Specified by:

getFavoredNodesManager in interface MasterServices

executePeerProcedure

private long executePeerProcedure(AbstractPeerProcedure<?> procedure)
                           throws IOException

Throws:

IOException

addReplicationPeer

public long addReplicationPeer(String peerId,
 ReplicationPeerConfig peerConfig,
 boolean enabled)
                        throws ReplicationException,
IOException

Description copied from interface: MasterServices

Add a new replication peer for replicating data to slave cluster

Specified by:

addReplicationPeer in interface MasterServices

Parameters:

peerId - a short name that identifies the peer

peerConfig - configuration for the replication slave cluster

enabled - peer state, true if ENABLED and false if DISABLED

Throws:

ReplicationException

IOException

removeReplicationPeer

public long removeReplicationPeer(String peerId)
                           throws ReplicationException,
IOException

Description copied from interface: MasterServices

Removes a peer and stops the replication

Specified by:

removeReplicationPeer in interface MasterServices

Parameters:

peerId - a short name that identifies the peer

Throws:

ReplicationException

IOException

enableReplicationPeer

public long enableReplicationPeer(String peerId)
                           throws ReplicationException,
IOException

Description copied from interface: MasterServices

Restart the replication stream to the specified peer

Specified by:

enableReplicationPeer in interface MasterServices

Parameters:

peerId - a short name that identifies the peer

Throws:

ReplicationException

IOException

disableReplicationPeer

public long disableReplicationPeer(String peerId)
                            throws ReplicationException,
IOException

Description copied from interface: MasterServices

Stop the replication stream to the specified peer

Specified by:

disableReplicationPeer in interface MasterServices

Parameters:

peerId - a short name that identifies the peer

Throws:

ReplicationException

IOException

getReplicationPeerConfig

public ReplicationPeerConfig getReplicationPeerConfig(String peerId)
                                               throws ReplicationException,
IOException

Description copied from interface: MasterServices

Returns the configured ReplicationPeerConfig for the specified peer

Specified by:

getReplicationPeerConfig in interface MasterServices

Parameters:

peerId - a short name that identifies the peer

Returns:

ReplicationPeerConfig for the peer

Throws:

ReplicationException

IOException

updateReplicationPeerConfig

public long updateReplicationPeerConfig(String peerId,
 ReplicationPeerConfig peerConfig)
                                 throws ReplicationException,
IOException

Description copied from interface: MasterServices

Update the peerConfig for the specified peer

Specified by:

updateReplicationPeerConfig in interface MasterServices

Parameters:

peerId - a short name that identifies the peer

peerConfig - new config for the peer

Throws:

ReplicationException

IOException

listReplicationPeers

public List<ReplicationPeerDescription> listReplicationPeers(String regex)
                                                      throws ReplicationException,
IOException

Description copied from interface: MasterServices

Return a list of replication peers.

Specified by:

listReplicationPeers in interface MasterServices

Parameters:

regex - The regular expression to match peer id

Returns:

a list of replication peers description

Throws:

ReplicationException

IOException

transitReplicationPeerSyncReplicationState

public long transitReplicationPeerSyncReplicationState(String peerId,
 SyncReplicationState state)
                                                throws ReplicationException,
IOException

Description copied from interface: MasterServices

Set current cluster state for a synchronous replication peer.

Specified by:

transitReplicationPeerSyncReplicationState in interface MasterServices

Parameters:

peerId - a short name that identifies the peer

state - state of current cluster

Throws:

ReplicationException

IOException

replicationPeerModificationSwitch

public boolean replicationPeerModificationSwitch(boolean on)
                                          throws IOException

Specified by:

replicationPeerModificationSwitch in interface MasterServices

Throws:

IOException

isReplicationPeerModificationEnabled

public boolean isReplicationPeerModificationEnabled()

Specified by:

isReplicationPeerModificationEnabled in interface MasterServices

decommissionRegionServers

public void decommissionRegionServers(List<ServerName> servers,
 boolean offload)
                               throws IOException

Mark region server(s) as decommissioned (previously called 'draining') to prevent additional regions from getting assigned to them. Also unload the regions on the servers asynchronously.0

Parameters:

servers - Region servers to decommission.

Throws:

IOException

listDecommissionedRegionServers

public List<ServerName> listDecommissionedRegionServers()

List region servers marked as decommissioned (previously called 'draining') to not get regions assigned to them.

Returns:

List of decommissioned servers.

recommissionRegionServer

public void recommissionRegionServer(ServerName server,
 List<byte[]> encodedRegionNames)
                              throws IOException

Remove decommission marker (previously called 'draining') from a region server to allow regions assignments. Load regions onto the server asynchronously if a list of regions is given

Parameters:

server - Region server to remove decommission marker from.

Throws:

IOException

getLockManager

public LockManager getLockManager()

Description copied from interface: MasterServices

Returns LockManager to lock namespaces/tables/regions.

Specified by:

getLockManager in interface MasterServices

getQuotaObserverChore

public QuotaObserverChore getQuotaObserverChore()

getSpaceQuotaSnapshotNotifier

public SpaceQuotaSnapshotNotifier getSpaceQuotaSnapshotNotifier()

getRemoteProcedure

private RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,?> getRemoteProcedure(long procId)

remoteProcedureCompleted

public void remoteProcedureCompleted(long procId,
 byte[] remoteResultData)

remoteProcedureFailed

public void remoteProcedureFailed(long procId,
 RemoteProcedureException error)

reopenRegions

long reopenRegions(TableName tableName,
 List<byte[]> regionNames,
 long nonceGroup,
 long nonce)
            throws IOException

Reopen regions provided in the argument

Parameters:

tableName - The current table name

regionNames - The region names of the regions to reopen

nonceGroup - Identifier for the source of the request, a client or process

nonce - A unique identifier for this operation from the client or process identified by nonceGroup (the source must ensure each operation gets a unique id).

Returns:

procedure Id

Throws:

IOException - if reopening region fails while running procedure

getReplicationPeerManager

public ReplicationPeerManager getReplicationPeerManager()

Description copied from interface: MasterServices

Returns the ReplicationPeerManager.

Specified by:

getReplicationPeerManager in interface MasterServices

getReplicationLogCleanerBarrier

public ReplicationLogCleanerBarrier getReplicationLogCleanerBarrier()

Description copied from interface: MasterServices

Returns the ReplicationLogCleanerBarrier. It will be used at multiple places so we put it in MasterServices directly.

Specified by:

getReplicationLogCleanerBarrier in interface MasterServices

getSyncReplicationPeerLock

public Semaphore getSyncReplicationPeerLock()

Description copied from interface: MasterServices

Returns the SyncReplicationPeerLock.

Specified by:

getSyncReplicationPeerLock in interface MasterServices

getReplicationLoad

public HashMap<String,List<Pair<ServerName,ReplicationLoadSource>>> getReplicationLoad(ServerName[] serverNames)

decorateMasterConfiguration

@Private
public static void decorateMasterConfiguration(org.apache.hadoop.conf.Configuration conf)

This method modifies the master's configuration in order to inject replication-related features

getSnapshotQuotaObserverChore

public SnapshotQuotaObserverChore getSnapshotQuotaObserverChore()

getActiveMasterManager

public ActiveMasterManager getActiveMasterManager()

getSyncReplicationReplayWALManager

public SyncReplicationReplayWALManager getSyncReplicationReplayWALManager()

Description copied from interface: MasterServices

Returns the SyncReplicationReplayWALManager.

Specified by:

getSyncReplicationReplayWALManager in interface MasterServices

getHbckChore

public HbckChore getHbckChore()

Description copied from interface: MasterServices

Returns Master's instance of HbckChore

Specified by:

getHbckChore in interface MasterServices

runReplicationBarrierCleaner

public void runReplicationBarrierCleaner()

Description copied from interface: MasterServices

Run the ReplicationBarrierChore.

Specified by:

runReplicationBarrierCleaner in interface MasterServices

getRSGroupInfoManager

public RSGroupInfoManager getRSGroupInfoManager()

Description copied from interface: MasterServices

Returns the RSGroupInfoManager

Specified by:

getRSGroupInfoManager in interface MasterServices

getCompactionState

public CompactionState getCompactionState(TableName tableName)

Get the compaction state of the table

Parameters:

tableName - The table name

Returns:

CompactionState Compaction state of the table

getMetaLocationSyncer

public MetaLocationSyncer getMetaLocationSyncer()

Description copied from interface: MasterServices

Get the meta location syncer.

We need to get this in MTP to tell the syncer the new meta replica count.

Specified by:

getMetaLocationSyncer in interface MasterServices

getMasterRegion

public MasterRegion getMasterRegion()

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration newConf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

createNamedQueueRecord

protected NamedQueueRecorder createNamedQueueRecord()

Specified by:

createNamedQueueRecord in class HBaseServerBase<MasterRpcServices>

clusterMode

protected boolean clusterMode()

Specified by:

clusterMode in class HBaseServerBase<MasterRpcServices>

getClusterId

public String getClusterId()

Description copied from interface: ConnectionRegistryEndpoint

Get cluster id.

Specified by:

getClusterId in interface ConnectionRegistryEndpoint

getActiveMaster

public Optional<ServerName> getActiveMaster()

Description copied from interface: ConnectionRegistryEndpoint

Get active master address.

Specified by:

getActiveMaster in interface ConnectionRegistryEndpoint

getBackupMasters

public List<ServerName> getBackupMasters()

Description copied from interface: ConnectionRegistryEndpoint

Get backup masters address.

Specified by:

getBackupMasters in interface ConnectionRegistryEndpoint

getBootstrapNodes

public Iterator<ServerName> getBootstrapNodes()

Description copied from interface: ConnectionRegistryEndpoint

Get a iterator of the available bootstrap nodes.

Specified by:

getBootstrapNodes in interface ConnectionRegistryEndpoint

getMetaLocations

public List<HRegionLocation> getMetaLocations()

Description copied from interface: ConnectionRegistryEndpoint

Get the location of meta regions.

Specified by:

getMetaLocations in interface ConnectionRegistryEndpoint

flushMasterStore

public void flushMasterStore()
                      throws IOException

Description copied from interface: MasterServices

Flush master local region

Specified by:

flushMasterStore in interface MasterServices

Throws:

IOException

getLiveRegionServers

public Collection<ServerName> getLiveRegionServers()

setLoadBalancer

void setLoadBalancer(RSGroupBasedLoadBalancer loadBalancer)

setAssignmentManager

void setAssignmentManager(AssignmentManager assignmentManager)

setDisableBalancerChoreForTest

static void setDisableBalancerChoreForTest(boolean disable)

setQuotasObserver

private void setQuotasObserver(org.apache.hadoop.conf.Configuration conf)

initializeCoprocessorHost

private void initializeCoprocessorHost(org.apache.hadoop.conf.Configuration conf)

flushTable

public long flushTable(TableName tableName,
 List<byte[]> columnFamilies,
 long nonceGroup,
 long nonce)
                throws IOException

Description copied from interface: MasterServices

Flush an existing table

Specified by:

flushTable in interface MasterServices

Parameters:

tableName - The table name

columnFamilies - The column families to flush

nonceGroup - the nonce group

nonce - the nonce

Returns:

the flush procedure id

Throws:

IOException

rollAllWALWriters

public long rollAllWALWriters(long nonceGroup,
 long nonce)
                       throws IOException

Description copied from interface: MasterServices

Roll WAL writer for all RegionServers

Specified by:

rollAllWALWriters in interface MasterServices

Returns:

procedure id

Throws:

IOException