Package org.apache.hadoop.hbase.regionserver

Class HRegionServer

java.lang.Object

java.lang.Thread

org.apache.hadoop.hbase.HBaseServerBase<RSRpcServices>

org.apache.hadoop.hbase.regionserver.HRegionServer

All Implemented Interfaces:

Runnable, Abortable, ConnectionRegistryEndpoint, ConfigurationObserver, FavoredNodesForRegion, LastSequenceId, MutableOnlineRegions, OnlineRegions, RegionServerServices, Server, Stoppable

Direct Known Subclasses:

MiniHBaseCluster.MiniHBaseClusterRegionServer

@LimitedPrivate("Tools")
public class HRegionServer
extends HBaseServerBase<RSRpcServices>
implements RegionServerServices, LastSequenceId

HRegionServer makes a set of HRegions available to clients. It checks in with the HMaster. There are many HRegionServers in a single HBase deployment.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	HRegionServer.CompactionChecker	Inner class that runs on a long period checking if regions need compaction.
private static class	HRegionServer.MovedRegionInfo
private static class	HRegionServer.PeriodicMemStoreFlusher
private static class	HRegionServer.SystemExitWhenAbortTimeout	Force to terminate region server when abort timeout.

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.regionserver.RegionServerServices

RegionServerServices.PostOpenDeployContext, RegionServerServices.RegionStateTransitionContext

Field Summary

Fields

Modifier and Type	Field	Description
(package private) static final String	ABORT_TIMEOUT
(package private) static final String	ABORT_TIMEOUT_TASK
private Timer	abortMonitor
private BlockCache	blockCache
private BootstrapNodeManager	bootstrapNodeManager
private BrokenStoreFileCleaner	brokenStoreFileCleaner
private MemStoreFlusher	cacheFlusher
private String	clusterId	Unique identifier for the cluster we are a part of.
(package private) CompactedHFilesDischarger	compactedFileDischarger
private ScheduledChore	compactionChecker	Check for compactions requests.
private final int	compactionCheckFrequency
private CompactSplit	compactSplitThread
private Map<String,org.apache.hbase.thirdparty.com.google.protobuf.Service>	coprocessorServiceHandlers
private boolean	dataFsOk
private static final long	DEFAULT_ABORT_TIMEOUT
private final org.apache.hbase.thirdparty.com.google.common.cache.Cache<Long,Long>	executedRegionProcedures	Used to cache the open/close region procedures which already executed.
private ExecutorStatusChore	executorStatusChore	The Executor status collect chore.
private final int	flushCheckFrequency
private ThroughputController	flushThroughputController
private FileSystemUtilizationChore	fsUtilizationChore
private HealthCheckChore	healthCheckChore	The health check chore.
private HeapMemoryManager	hMemManager
private boolean	killed
private LeaseManager	leaseManager
private org.apache.hadoop.hbase.shaded.protobuf.generated.LockServiceProtos.LockService.BlockingInterface	lockStub
private static final org.slf4j.Logger	LOG
private final MasterAddressTracker	masterAddressTracker
private final boolean	masterless	True if this RegionServer is coming up in a cluster where there is no Master; means it needs to just come up and make do without a Master to talk to: e.g.
private static final String	MASTERLESS_CONFIG_NAME
private MetricsRegionServer	metricsRegionServer
(package private) MetricsRegionServerWrapperImpl	metricsRegionServerImpl
private MobFileCache	mobFileCache
private final org.apache.hbase.thirdparty.com.google.common.cache.Cache<String,HRegionServer.MovedRegionInfo>	movedRegionInfoCache	Used to cache the moved-out regions
private NamedQueueServiceChore	namedQueueServiceChore
(package private) final ServerNonceManager	nonceManager	Nonce manager.
private ScheduledChore	nonceManagerChore	The nonce manager chore.
(package private) final AtomicBoolean	online
private final Map<String,HRegion>	onlineRegions	Map of regions currently being served by this region server.
private final ReentrantReadWriteLock	onlineRegionsLock	Lock for gating access to onlineRegions.
private JvmPauseMonitor	pauseMonitor
private static final String	PERIOD_COMPACTION
private static final String	PERIOD_FLUSH
private ScheduledChore	periodicFlusher	Check for flushes
private PrefetchExecutorNotifier	prefetchExecutorNotifier
private RemoteProcedureResultReporter	procedureResultReporter
private final Map<String,Address[]>	regionFavoredNodesMap	Map of encoded region names to the DataNode locations they should be hosted on We store the value as Address since InetSocketAddress is required by the HDFS API (create() that takes favored nodes as hints for placing file blocks).
private RegionReplicationBufferManager	regionReplicationBufferManager
static final String	REGIONSERVER	region server process name
private static final String	REGIONSERVER_CODEC	regionserver codec list
private final RegionServerAccounting	regionServerAccounting
private final ConcurrentMap<byte[],Boolean>	regionsInTransitionInRS	A map from RegionName to current action in progress.
private ReplicationMarkerChore	replicationMarkerChore
private ReplicationSinkService	replicationSinkHandler
private ReplicationSourceService	replicationSourceHandler
private final long	retryPauseTime
private RpcClient	rpcClient
(package private) static final String	RS_HOSTNAME_DISABLE_MASTER_REVERSEDNS_KEY	Deprecated. since 2.4.0 and will be removed in 4.0.0.
private RegionServerCoprocessorHost	rsHost
private RSMobFileCleanerChore	rsMobFileCleanerChore
private RegionServerProcedureManagerHost	rspmHost
private RegionServerRpcQuotaManager	rsQuotaManager
private RSSnapshotVerifier	rsSnapshotVerifier
private RegionServerSpaceQuotaManager	rsSpaceQuotaManager
private org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStatusService.BlockingInterface	rssStub
private boolean	sameReplicationSourceAndSink
private SecureBulkLoadManager	secureBulkLoadManager
private final int	shortOperationTimeout
private SplitLogWorker	splitLogWorker
private boolean	stopping
private StorefileRefresherChore	storefileRefresher
private final ConcurrentMap<Long,Long>	submittedRegionProcedures	Used to cache the open/close region procedures which already submitted.
static boolean	TEST_SKIP_REPORTING_TRANSITION	For testing only! Set to true to skip notifying region assignment to master .
private final int	threadWakeFrequency
private static final int	TIMEOUT_REGION_MOVED	We need a timeout.
private Thread.UncaughtExceptionHandler	uncaughtExceptionHandler
(package private) int	unitKB
(package private) int	unitMB
(package private) static final String	UNSAFE_RS_HOSTNAME_DISABLE_MASTER_REVERSEDNS_KEY	HBASE-18226: This config and hbase.unsafe.regionserver.hostname are mutually exclusive.
private WALFactory	walFactory
private LogRoller	walRoller

Fields inherited from class org.apache.hadoop.hbase.HBaseServerBase

abortRequested, asyncClusterConnection, choreService, clusterStatusTracker, conf, configurationManager, csm, dataFs, dataRootDir, eventLoopGroupConfig, executorService, infoServer, metaRegionLocationCache, msgInterval, namedQueueRecorder, rpcServices, serverName, sleeper, startcode, stopped, tableDescriptors, userProvider, useThisHostnameInstead, walFs, walRootDir, zooKeeper

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor	Description
HRegionServer(org.apache.hadoop.conf.Configuration conf)	Starts a HRegionServer at the default location.

Method Summary

Modifier and Type	Method	Description
void	abort(String reason, Throwable cause)	Cause the server to exit without closing the regions it is serving, the log it is using and without notifying the master.
private void	addRegion(SortedMap<Long,Collection<HRegion>> sortedRegions, HRegion region, long size)
void	addRegion(HRegion region)	Add to online regions.
private void	addToMovedRegions(String encodedName, ServerName destination, long closeSeqNum, boolean selfMove)
private boolean	areAllUserRegionsOffline()
private void	blockAndCheckIfStopped(ZKNodeTracker tracker)	Utilty method to wait indefinitely on a znode availability while checking if the region server is shut down
(package private) org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionSpaceUseReportRequest	buildRegionSpaceUseReportRequest(RegionSizeStore regionSizes)	Builds a RegionServerStatusProtos.RegionSpaceUseReportRequest protobuf message from the region size map.
private void	buildReportAndSend(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStatusService.BlockingInterface rss, RegionSizeStore regionSizeStore)	Builds the region size report and sends it to the master.
private org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.ServerLoad	buildServerLoad(long reportStartTime, long reportEndTime)
protected boolean	cacheTableDescriptor()
protected boolean	canCreateBaseZNode()
protected boolean	canUpdateTableDescriptor()
private static void	checkCodecs(org.apache.hadoop.conf.Configuration c)	Run test on configured codecs to make sure supporting libs are in place.
(package private) boolean	checkFileSystem()	Checks to see if the file system is still accessible.
private Throwable	cleanup(Throwable t, String msg)	Cleanup after Throwable caught invoking method.
(package private) CacheEvictionStats	clearRegionBlockCache(Region region)
private void	closeMetaTableRegions(boolean abort)	Close meta region if we carry it
protected boolean	closeRegion(String encodedName, boolean abort, ServerName destination)	Close asynchronously a region, can be called from the master or internally by the regionserver when stopping.
private void	closeRegionIgnoreErrors(RegionInfo region, boolean abort)	Try to close the region, logs a warning on failure but continues.
private void	closeUserRegions(boolean abort)	Schedule closes on all user regions.
protected boolean	clusterMode()
private void	computeIfPersistentBucketCache(Consumer<BucketCache> computation)
protected void	configureInfoServer(InfoServer infoServer)
(package private) static HRegionServer	constructRegionServer(Class<? extends HRegionServer> regionServerClass, org.apache.hadoop.conf.Configuration conf)	Utility for constructing an instance of the passed HRegionServer class.
private boolean	containsMetaTableRegions()
(package private) org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionSpaceUse	convertRegionSize(RegionInfo regionInfo, Long sizeInBytes)	Converts a pair of RegionInfo and long into a RegionServerStatusProtos.RegionSpaceUse protobuf message.
private IOException	convertThrowableToIOE(Throwable t, String msg)
private void	createMyEphemeralNode()
protected NamedQueueRecorder	createNamedQueueRecord()
private static void	createNewReplicationInstance(org.apache.hadoop.conf.Configuration conf, HRegionServer server, org.apache.hadoop.fs.FileSystem walFs, org.apache.hadoop.fs.Path walDir, org.apache.hadoop.fs.Path oldWALDir, WALFactory walFactory)	Load the replication executorService objects, if any
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad	createRegionLoad(String encodedRegionName)
(package private) org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad	createRegionLoad(HRegion r, org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad.Builder regionLoadBldr, org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier.Builder regionSpecifier)
private ServerName	createRegionServerStatusStub()	Get the current master from ZooKeeper and open the RPC connection to it.
protected ServerName	createRegionServerStatusStub(boolean refresh)	Get the current master from ZooKeeper and open the RPC connection to it.
private org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionRequest	createReportRegionStateTransitionRequest(RegionServerServices.RegionStateTransitionContext context)
protected RSRpcServices	createRpcServices()
private org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.UserLoad	createUserLoad(String user, MetricsUserSource userSource)
private void	deleteMyEphemeralNode()
void	dumpRowLocks(PrintWriter out)	Used by RSDumpServlet to generate debugging information.
(package private) org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.CoprocessorServiceResponse	execRegionServerService(org.apache.hbase.thirdparty.com.google.protobuf.RpcController controller, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.CoprocessorServiceRequest serviceRequest)
(package private) void	executeProcedure(long procId, long initiatingMasterActiveTime, RSProcedureCallable callable)
void	finishRegionProcedure(long procId)	See submitRegionProcedure(long).
Optional<ServerName>	getActiveMaster()	Get active master address.
List<ServerName>	getBackupMasters()	Get backup masters address.
Optional<BlockCache>	getBlockCache()	May be null if this is a master which not carry table.
Iterator<ServerName>	getBootstrapNodes()	Get a iterator of the available bootstrap nodes.
BrokenStoreFileCleaner	getBrokenStoreFileCleaner()
String	getClusterId()	Get cluster id.
CompactedHFilesDischarger	getCompactedHFilesDischarger()
double	getCompactionPressure()
CompactionRequester	getCompactionRequestor()
CompactSplit	getCompactSplitThread()	Returns the underlying CompactSplit for the servers
protected RegionServerCoprocessorHost	getCoprocessorHost()
(package private) SortedMap<Long,Collection<HRegion>>	getCopyOfOnlineRegionsSortedByOffHeapSize()
(package private) SortedMap<Long,Collection<HRegion>>	getCopyOfOnlineRegionsSortedByOnHeapSize()
protected Class<? extends javax.servlet.http.HttpServlet>	getDumpServlet()
InetSocketAddress[]	getFavoredNodesForRegion(String encodedRegionName)	Return the favored nodes for a region given its encoded name.
double	getFlushPressure()
FlushRequester	getFlushRequester()	Returns reference to FlushRequester
ThroughputController	getFlushThroughputController()	Returns the controller to avoid flush too fast
HeapMemoryManager	getHeapMemoryManager()	Returns heap memory manager instance
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionStoreSequenceIds	getLastSequenceId(byte[] encodedRegionName)
LeaseManager	getLeaseManager()	Returns The RegionServer's "Leases" service
MasterAddressTracker	getMasterAddressTracker()	Returns Master address tracker instance.
MemStoreFlusher	getMemStoreFlusher()
List<HRegionLocation>	getMetaLocations()	Get the location of meta regions.
MetricsRegionServer	getMetrics()	Returns the metrics tracker for the region server
Optional<MobFileCache>	getMobFileCache()	May be null if this is a master which not carry table.
HRegionServer.MovedRegionInfo	getMovedRegion(String encodedRegionName)
private String	getMyEphemeralNodePath()
ServerNonceManager	getNonceManager()	Only required for "old" log replay; if it's removed, remove this.
int	getNumberOfOnlineRegions()
HRegion	getOnlineRegion(byte[] regionName)
protected Map<String,HRegion>	getOnlineRegions()
private String	getOnlineRegionsAsPrintableString()
Collection<HRegion>	getOnlineRegionsLocalContext()	For tests, web ui and metrics.
Set<TableName>	getOnlineTables()	Gets the online tables in this RS.
protected String	getProcessName()
protected HRegion	getRegion(byte[] regionName)	Protected Utility method for safely obtaining an HRegion handle.
HRegion	getRegion(String encodedRegionName)	Return Region instance.
private HRegion	getRegionByEncodedName(byte[] regionName, String encodedRegionName)
HRegion	getRegionByEncodedName(String encodedRegionName)
RegionReplicationBufferManager	getRegionReplicationBufferManager()
List<HRegion>	getRegions()	Get all online regions in this RS.
List<HRegion>	getRegions(TableName tableName)	Gets the online regions of the specified table.
RegionServerAccounting	getRegionServerAccounting()	Returns the RegionServerAccounting for this Region Server
RegionServerCoprocessorHost	getRegionServerCoprocessorHost()
String[]	getRegionServerCoprocessors()
RegionServerRpcQuotaManager	getRegionServerRpcQuotaManager()	Returns RegionServer's instance of RegionServerRpcQuotaManager
RegionServerSpaceQuotaManager	getRegionServerSpaceQuotaManager()	Returns RegionServer's instance of RegionServerSpaceQuotaManager
ConcurrentMap<byte[],Boolean>	getRegionsInTransitionInRS()	Get the regions that are currently being opened or closed in the RS
ReplicationSinkService	getReplicationSinkService()	Returns Return the object that implements the replication sink executorService.
ReplicationSourceService	getReplicationSourceService()	Returns Return the object that implements the replication source executorService.
long	getRetryPauseTime()	Return pause time configured in HConstants.HBASE_RPC_SHORTOPERATION_RETRY_PAUSE_TIME}
RSMobFileCleanerChore	getRSMobFileCleanerChore()
RSRpcServices	getRSRpcServices()
(package private) RSSnapshotVerifier	getRsSnapshotVerifier()
SecureBulkLoadManager	getSecureBulkLoadManager()	Returns RegionServer's instance of SecureBulkLoadManager
protected String	getUseThisHostnameInstead(org.apache.hadoop.conf.Configuration conf)
WAL	getWAL(RegionInfo regionInfo)	Returns the WAL for a particular region.
private WALActionsListener	getWALEventTrackerListener(org.apache.hadoop.conf.Configuration conf)
WALFactory	getWalFactory()
Map<String,ReplicationStatus>	getWalGroupsReplicationStatus()
LogRoller	getWalRoller()
List<WAL>	getWALs()	Returns the List of WALs that are used by this server Doesn't include the meta WAL
private long	getWriteRequestCount()	Returns Current write count for all online regions.
protected void	handleReportForDutyResponse(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStartupResponse c)	Run init.
private void	initializeReplicationMarkerChore()
private void	initializeThreads()
private void	initializeZooKeeper()	Bring up connection to zk ensemble and then wait until a master for this cluster and then after that, wait until cluster 'up' flag has been set.
boolean	isClusterUp()	Returns True if the cluster is up.
(package private) boolean	isDataFileSystemOk()	Returns true when the data file system is available, false otherwise.
private boolean	isHealthCheckerConfigured()
private boolean	isHealthy()
boolean	isOnline()	Report the status of the server.
boolean	isStopping()	Returns True is the server is Stopping
private boolean	keepLooping()
protected void	kill()
protected void	login(UserProvider user, String host)
static void	main(String[] args)
int	movedRegionCacheExpiredTime()
private static <T extends ReplicationService> T	newReplicationInstance(String classname, Class<T> xface, org.apache.hadoop.conf.Configuration conf, HRegionServer server, org.apache.hadoop.fs.FileSystem walFs, org.apache.hadoop.fs.Path logDir, org.apache.hadoop.fs.Path oldLogDir, WALFactory walFactory)
void	onConfigurationChange(org.apache.hadoop.conf.Configuration newConf)	This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
void	postOpenDeployTasks(RegionServerServices.PostOpenDeployContext context)	Tasks to perform after region open to complete deploy of region on regionserver
private void	preRegistrationInitialization()	All initialization needed before we go register with Master. Do bare minimum.
EntityLock	regionLock(List<RegionInfo> regionInfo, String description, Abortable abort)	Master based locks on namespaces/tables/regions.
private void	registerConfigurationObservers()
boolean	registerService(org.apache.hbase.thirdparty.com.google.protobuf.Service instance)	Registers a new protocol buffer Service subclass as a coprocessor endpoint to be available for handling
void	remoteProcedureComplete(long procId, long initiatingMasterActiveTime, Throwable error)
(package private) void	removeFromMovedRegions(String encodedName)
boolean	removeRegion(HRegion r, ServerName destination)	Removes the given Region from the list of onlineRegions.
boolean	reportFileArchivalForQuotas(TableName tableName, Collection<Map.Entry<String,Long>> archivedFiles)	Reports a collection of files, and their sizes, that belonged to the given table were just moved to the archive directory.
private org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStartupResponse	reportForDuty()
(package private) void	reportProcedureDone(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportProcedureDoneRequest request)
boolean	reportRegionSizesForQuotas(RegionSizeStore regionSizeStore)	Reports the given map of Regions and their size on the filesystem to the active Master.
boolean	reportRegionStateTransition(RegionServerServices.RegionStateTransitionContext context)	Notify master that a handler requests to change a region state
private static int	roundSize(long sizeInByte, int sizeUnit)
void	run()	The HRegionServer sticks in this loop until closed.
private void	scheduleAbortTimer()
private void	setupWALAndReplication()	Setup WAL log and replication if enabled.
private void	shutdownWAL(boolean close)
private boolean	skipReportingTransition(RegionServerServices.RegionStateTransitionContext context)	Helper method for use in tests.
private static boolean	sleepInterrupted(long millis)
private void	startHeapMemoryManager()
private void	startReplicationService()	Start up replication source and sink handlers.
private void	startServices()	Start maintenance Threads, Server, Worker and lease checker threads.
void	stop(String msg)	Stop this service.
void	stop(String msg, boolean force, User user)	Stops the regionserver.
protected void	stopChores()
protected void	stopServiceThreads()	Wait on all threads to finish.
(package private) boolean	submitRegionProcedure(long procId)	Will ignore the open/close region procedures which already submitted or executed.
private void	triggerFlushInPrimaryRegion(HRegion region)	Trigger a flush in the primary region replica if this region is a secondary replica.
protected void	tryRegionServerReport(long reportStartTime, long reportEndTime)
void	unassign(byte[] regionName)	Unassign the given region from the current regionserver and assign it randomly.
void	updateRegionFavoredNodesMapping(String encodedRegionName, List<org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ServerName> favoredNodes)	Used to update the favored nodes mapping when required.
void	waitForServerOnline()
private void	waitOnAllRegionsToClose(boolean abort)	Wait on regions close.
boolean	walRollRequestFinished()	For testing

Methods inherited from class org.apache.hadoop.hbase.HBaseServerBase

closeClusterConnection, closeTableDescriptors, closeZooKeeper, createConnection, getAccessChecker, getAsyncClusterConnection, getChoreService, getConfiguration, getConfigurationManager, getCoordinatedStateManager, getDataRootDir, getEventLoopGroupConfig, getExecutorService, getFileSystem, getInfoServer, getMetaRegionLocationCache, getMsgInterval, getNamedQueueRecorder, getRpcServer, getRpcServices, getServerName, getStartcode, getTableDescriptors, getWALFileSystem, getWALRootDir, getZKPermissionWatcher, getZooKeeper, initializeFileSystem, initializeMemStoreChunkCreator, installShutdownHook, isAborted, isShutdownHookInstalled, isStopped, setAbortRequested, setupClusterConnection, shutdownChore, stopChoreService, stopExecutorService, stopInfoServer, toString, updateConfiguration

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, suspend, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.Abortable

abort, isAborted

Methods inherited from interface org.apache.hadoop.hbase.regionserver.RegionServerServices

getAccessChecker, getExecutorService, getRpcServer, getTableDescriptors, getZKPermissionWatcher

Methods inherited from interface org.apache.hadoop.hbase.Server

createConnection, getAsyncClusterConnection, getAsyncConnection, getChoreService, getConfiguration, getConnection, getCoordinatedStateManager, getFileSystem, getServerName, getZooKeeper

Methods inherited from interface org.apache.hadoop.hbase.Stoppable

isStopped

Field Details

LOG

private static final org.slf4j.Logger LOG

unitMB

int unitMB

unitKB

int unitKB

TEST_SKIP_REPORTING_TRANSITION

@Private
public static boolean TEST_SKIP_REPORTING_TRANSITION

For testing only! Set to true to skip notifying region assignment to master .

regionsInTransitionInRS

private final ConcurrentMap<byte[],Boolean> regionsInTransitionInRS

A map from RegionName to current action in progress. Boolean value indicates: true - if open region action in progress false - if close region action in progress

submittedRegionProcedures

private final ConcurrentMap<Long,Long> submittedRegionProcedures

Used to cache the open/close region procedures which already submitted. See submitRegionProcedure(long).

executedRegionProcedures

private final org.apache.hbase.thirdparty.com.google.common.cache.Cache<Long,Long> executedRegionProcedures

Used to cache the open/close region procedures which already executed. See submitRegionProcedure(long).

movedRegionInfoCache

private final org.apache.hbase.thirdparty.com.google.common.cache.Cache<String,HRegionServer.MovedRegionInfo> movedRegionInfoCache

Used to cache the moved-out regions

cacheFlusher

private MemStoreFlusher cacheFlusher

hMemManager

private HeapMemoryManager hMemManager

replicationSourceHandler

private ReplicationSourceService replicationSourceHandler

replicationSinkHandler

private ReplicationSinkService replicationSinkHandler

sameReplicationSourceAndSink

private boolean sameReplicationSourceAndSink

compactSplitThread

private CompactSplit compactSplitThread

onlineRegions

private final Map<String,HRegion> onlineRegions

Map of regions currently being served by this region server. Key is the encoded region name. All access should be synchronized.

onlineRegionsLock

private final ReentrantReadWriteLock onlineRegionsLock

Lock for gating access to onlineRegions. TODO: If this map is gated by a lock, does it need to be a ConcurrentHashMap?

regionFavoredNodesMap

private final Map<String,Address[]> regionFavoredNodesMap

Map of encoded region names to the DataNode locations they should be hosted on We store the value as Address since InetSocketAddress is required by the HDFS API (create() that takes favored nodes as hints for placing file blocks). We could have used ServerName here as the value class, but we'd need to convert it to InetSocketAddress at some point before the HDFS API call, and it seems a bit weird to store ServerName since ServerName refers to RegionServers and here we really mean DataNode locations. We don't store it as InetSocketAddress here because the conversion on demand from Address to InetSocketAddress will guarantee the resolution results will be fresh when we need it.

leaseManager

private LeaseManager leaseManager

dataFsOk

private volatile boolean dataFsOk

ABORT_TIMEOUT

static final String ABORT_TIMEOUT

See Also:

• Constant Field Values

DEFAULT_ABORT_TIMEOUT

private static final long DEFAULT_ABORT_TIMEOUT

See Also:

• Constant Field Values

ABORT_TIMEOUT_TASK

static final String ABORT_TIMEOUT_TASK

See Also:

• Constant Field Values

stopping

private boolean stopping

killed

private volatile boolean killed

threadWakeFrequency

private final int threadWakeFrequency

PERIOD_COMPACTION

private static final String PERIOD_COMPACTION

See Also:

• Constant Field Values

compactionCheckFrequency

private final int compactionCheckFrequency

PERIOD_FLUSH

private static final String PERIOD_FLUSH

See Also:

• Constant Field Values

flushCheckFrequency

private final int flushCheckFrequency

rssStub

private volatile org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStatusService.BlockingInterface rssStub

lockStub

private volatile org.apache.hadoop.hbase.shaded.protobuf.generated.LockServiceProtos.LockService.BlockingInterface lockStub

rpcClient

private RpcClient rpcClient

uncaughtExceptionHandler

private Thread.UncaughtExceptionHandler uncaughtExceptionHandler

pauseMonitor

private JvmPauseMonitor pauseMonitor

rsSnapshotVerifier

private RSSnapshotVerifier rsSnapshotVerifier

REGIONSERVER

public static final String REGIONSERVER

region server process name

See Also:

• Constant Field Values

metricsRegionServer

private MetricsRegionServer metricsRegionServer

metricsRegionServerImpl

MetricsRegionServerWrapperImpl metricsRegionServerImpl

compactionChecker

private ScheduledChore compactionChecker

Check for compactions requests.

periodicFlusher

private ScheduledChore periodicFlusher

Check for flushes

walFactory

private volatile WALFactory walFactory

walRoller

private LogRoller walRoller

procedureResultReporter

private RemoteProcedureResultReporter procedureResultReporter

online

final AtomicBoolean online

masterAddressTracker

private final MasterAddressTracker masterAddressTracker

splitLogWorker

private SplitLogWorker splitLogWorker

shortOperationTimeout

private final int shortOperationTimeout

retryPauseTime

private final long retryPauseTime

regionServerAccounting

private final RegionServerAccounting regionServerAccounting

namedQueueServiceChore

private NamedQueueServiceChore namedQueueServiceChore

blockCache

private BlockCache blockCache

mobFileCache

private MobFileCache mobFileCache

healthCheckChore

private HealthCheckChore healthCheckChore

The health check chore.

executorStatusChore

private ExecutorStatusChore executorStatusChore

The Executor status collect chore.

nonceManagerChore

private ScheduledChore nonceManagerChore

The nonce manager chore.

coprocessorServiceHandlers

private Map<String,org.apache.hbase.thirdparty.com.google.protobuf.Service> coprocessorServiceHandlers

RS_HOSTNAME_DISABLE_MASTER_REVERSEDNS_KEY

@Deprecated
@LimitedPrivate("Configuration")
static final String RS_HOSTNAME_DISABLE_MASTER_REVERSEDNS_KEY

Deprecated.

since 2.4.0 and will be removed in 4.0.0. Use UNSAFE_RS_HOSTNAME_DISABLE_MASTER_REVERSEDNS_KEY instead.

See Also:

• HBASE-24667
• Constant Field Values

UNSAFE_RS_HOSTNAME_DISABLE_MASTER_REVERSEDNS_KEY

@LimitedPrivate("Configuration")
static final String UNSAFE_RS_HOSTNAME_DISABLE_MASTER_REVERSEDNS_KEY

HBASE-18226: This config and hbase.unsafe.regionserver.hostname are mutually exclusive. Exception will be thrown if both are used.

See Also:

• Constant Field Values

clusterId

private String clusterId

Unique identifier for the cluster we are a part of.

storefileRefresher

private StorefileRefresherChore storefileRefresher

rsHost

private volatile RegionServerCoprocessorHost rsHost

rspmHost

private RegionServerProcedureManagerHost rspmHost

rsQuotaManager

private RegionServerRpcQuotaManager rsQuotaManager

rsSpaceQuotaManager

private RegionServerSpaceQuotaManager rsSpaceQuotaManager

nonceManager

final ServerNonceManager nonceManager

Nonce manager. Nonces are used to make operations like increment and append idempotent in the case where client doesn't receive the response from a successful operation and retries. We track the successful ops for some time via a nonce sent by client and handle duplicate operations (currently, by failing them; in future we might use MVCC to return result). Nonces are also recovered from WAL during, recovery; however, the caveats (from HBASE-3787) are: - WAL recovery is optimized, and under high load we won't read nearly nonce-timeout worth of past records. If we don't read the records, we don't read and recover the nonces. Some WALs within nonce-timeout at recovery may not even be present due to rolling/cleanup. - There's no WAL recovery during normal region move, so nonces will not be transfered. We can have separate additional "Nonce WAL". It will just contain bunch of numbers and won't be flushed on main path - because WAL itself also contains nonces, if we only flush it before memstore flush, for a given nonce we will either see it in the WAL (if it was never flushed to disk, it will be part of recovery), or we'll see it as part of the nonce log (or both occasionally, which doesn't matter). Nonce log file can be deleted after the latest nonce in it expired. It can also be recovered during move.

brokenStoreFileCleaner

private BrokenStoreFileCleaner brokenStoreFileCleaner

rsMobFileCleanerChore

private RSMobFileCleanerChore rsMobFileCleanerChore

compactedFileDischarger

@Private
CompactedHFilesDischarger compactedFileDischarger

flushThroughputController

private volatile ThroughputController flushThroughputController

secureBulkLoadManager

private SecureBulkLoadManager secureBulkLoadManager

fsUtilizationChore

private FileSystemUtilizationChore fsUtilizationChore

bootstrapNodeManager

private BootstrapNodeManager bootstrapNodeManager

masterless

private final boolean masterless

True if this RegionServer is coming up in a cluster where there is no Master; means it needs to just come up and make do without a Master to talk to: e.g. in test or HRegionServer is doing other than its usual duties: e.g. as an hollowed-out host whose only purpose is as a Replication-stream sink; see HBASE-18846 for more. TODO: can this replace TEST_SKIP_REPORTING_TRANSITION ?

MASTERLESS_CONFIG_NAME

private static final String MASTERLESS_CONFIG_NAME

See Also:

• Constant Field Values

REGIONSERVER_CODEC

private static final String REGIONSERVER_CODEC

regionserver codec list

See Also:

• Constant Field Values

abortMonitor

private Timer abortMonitor

regionReplicationBufferManager

private RegionReplicationBufferManager regionReplicationBufferManager

replicationMarkerChore

private ReplicationMarkerChore replicationMarkerChore

prefetchExecutorNotifier

private PrefetchExecutorNotifier prefetchExecutorNotifier

TIMEOUT_REGION_MOVED

private static final int TIMEOUT_REGION_MOVED

We need a timeout. If not there is a risk of giving a wrong information: this would double the number of network calls instead of reducing them.

See Also:

• Constant Field Values

Constructor Details

HRegionServer

public HRegionServer(org.apache.hadoop.conf.Configuration conf)
              throws IOException

Starts a HRegionServer at the default location.

Don't start any services or managers in here in the Constructor. Defer till after we register with the Master as much as possible. See startServices().

Throws:

IOException

Method Details

getUseThisHostnameInstead

protected String getUseThisHostnameInstead(org.apache.hadoop.conf.Configuration conf)
                                    throws IOException

Specified by:

getUseThisHostnameInstead in class HBaseServerBase<RSRpcServices>

Throws:

IOException

login

protected void login(UserProvider user,
 String host)
              throws IOException

Specified by:

login in class HBaseServerBase<RSRpcServices>

Throws:

IOException

getProcessName

protected String getProcessName()

Specified by:

getProcessName in class HBaseServerBase<RSRpcServices>

getCoprocessorHost

protected RegionServerCoprocessorHost getCoprocessorHost()

Specified by:

getCoprocessorHost in class HBaseServerBase<RSRpcServices>

canCreateBaseZNode

protected boolean canCreateBaseZNode()

Specified by:

canCreateBaseZNode in class HBaseServerBase<RSRpcServices>

canUpdateTableDescriptor

protected boolean canUpdateTableDescriptor()

Specified by:

canUpdateTableDescriptor in class HBaseServerBase<RSRpcServices>

cacheTableDescriptor

protected boolean cacheTableDescriptor()

Specified by:

cacheTableDescriptor in class HBaseServerBase<RSRpcServices>

createRpcServices

protected RSRpcServices createRpcServices()
                                   throws IOException

Specified by:

createRpcServices in class HBaseServerBase<RSRpcServices>

Throws:

IOException

configureInfoServer

protected void configureInfoServer(InfoServer infoServer)

Specified by:

configureInfoServer in class HBaseServerBase<RSRpcServices>

getDumpServlet

protected Class<? extends javax.servlet.http.HttpServlet> getDumpServlet()

Specified by:

getDumpServlet in class HBaseServerBase<RSRpcServices>

dumpRowLocks

public void dumpRowLocks(PrintWriter out)

Used by RSDumpServlet to generate debugging information.

registerService

public boolean registerService(org.apache.hbase.thirdparty.com.google.protobuf.Service instance)

Description copied from interface: RegionServerServices

Registers a new protocol buffer Service subclass as a coprocessor endpoint to be available for handling

Specified by:

registerService in interface RegionServerServices

Parameters:

instance - the Service subclass instance to expose as a coprocessor endpoint

Returns:

true if the registration was successful, false

checkCodecs

private static void checkCodecs(org.apache.hadoop.conf.Configuration c)
                         throws IOException

Run test on configured codecs to make sure supporting libs are in place.

Throws:

IOException

getClusterId

public String getClusterId()

Description copied from interface: ConnectionRegistryEndpoint

Get cluster id.

Specified by:

getClusterId in interface ConnectionRegistryEndpoint

preRegistrationInitialization

private void preRegistrationInitialization()

All initialization needed before we go register with Master.
Do bare minimum. Do bulk of initializations AFTER we've connected to the Master.
In here we just put up the RpcServer, setup Connection, and ZooKeeper.

initializeZooKeeper

private void initializeZooKeeper()
                          throws IOException,
InterruptedException

Bring up connection to zk ensemble and then wait until a master for this cluster and then after that, wait until cluster 'up' flag has been set. This is the order in which master does things.

Finally open long-living server short-circuit connection.

Throws:

IOException

InterruptedException

blockAndCheckIfStopped

private void blockAndCheckIfStopped(ZKNodeTracker tracker)
                             throws IOException,
InterruptedException

Utilty method to wait indefinitely on a znode availability while checking if the region server is shut down

Parameters:

tracker - znode tracker to use

Throws:

IOException - any IO exception, plus if the RS is stopped

InterruptedException - if the waiting thread is interrupted

isClusterUp

public boolean isClusterUp()

Returns True if the cluster is up.

Specified by:

isClusterUp in interface RegionServerServices

Overrides:

isClusterUp in class HBaseServerBase<RSRpcServices>

initializeReplicationMarkerChore

private void initializeReplicationMarkerChore()

isStopping

public boolean isStopping()

Description copied from interface: Server

Returns True is the server is Stopping

Specified by:

isStopping in interface Server

run

public void run()

The HRegionServer sticks in this loop until closed.

Specified by:

run in interface Runnable

Overrides:

run in class Thread

containsMetaTableRegions

private boolean containsMetaTableRegions()

areAllUserRegionsOffline

private boolean areAllUserRegionsOffline()

getWriteRequestCount

private long getWriteRequestCount()

Returns Current write count for all online regions.

tryRegionServerReport

@Private
protected void tryRegionServerReport(long reportStartTime,
 long reportEndTime)
                              throws IOException

Throws:

IOException

reportRegionSizesForQuotas

public boolean reportRegionSizesForQuotas(RegionSizeStore regionSizeStore)

Reports the given map of Regions and their size on the filesystem to the active Master.

Specified by:

reportRegionSizesForQuotas in interface RegionServerServices

Parameters:

regionSizeStore - The store containing region sizes

Returns:

false if FileSystemUtilizationChore should pause reporting to master. true otherwise

buildReportAndSend

private void buildReportAndSend(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStatusService.BlockingInterface rss,
 RegionSizeStore regionSizeStore)
                         throws org.apache.hbase.thirdparty.com.google.protobuf.ServiceException

Builds the region size report and sends it to the master. Upon successful sending of the report, the region sizes that were sent are marked as sent.

Parameters:

rss - The stub to send to the Master

regionSizeStore - The store containing region sizes

Throws:

org.apache.hbase.thirdparty.com.google.protobuf.ServiceException

buildRegionSpaceUseReportRequest

org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionSpaceUseReportRequest buildRegionSpaceUseReportRequest(RegionSizeStore regionSizes)

Builds a RegionServerStatusProtos.RegionSpaceUseReportRequest protobuf message from the region size map.

Parameters:

regionSizes - The size in bytes of regions

Returns:

The corresponding protocol buffer message.

convertRegionSize

org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionSpaceUse convertRegionSize(RegionInfo regionInfo,
 Long sizeInBytes)

Converts a pair of RegionInfo and long into a RegionServerStatusProtos.RegionSpaceUse protobuf message.

Parameters:

regionInfo - The RegionInfo

sizeInBytes - The size in bytes of the Region

Returns:

The protocol buffer

buildServerLoad

private org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.ServerLoad buildServerLoad(long reportStartTime,
 long reportEndTime)
                                                                                                  throws IOException

Throws:

IOException

getOnlineRegionsAsPrintableString

private String getOnlineRegionsAsPrintableString()

waitOnAllRegionsToClose

private void waitOnAllRegionsToClose(boolean abort)

Wait on regions close.

sleepInterrupted

private static boolean sleepInterrupted(long millis)

shutdownWAL

private void shutdownWAL(boolean close)

handleReportForDutyResponse

protected void handleReportForDutyResponse(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStartupResponse c)
                                    throws IOException

Run init. Sets up wal and starts up all server threads.

Parameters:

c - Extra configuration.

Throws:

IOException

startHeapMemoryManager

private void startHeapMemoryManager()

createMyEphemeralNode

private void createMyEphemeralNode()
                            throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

deleteMyEphemeralNode

private void deleteMyEphemeralNode()
                            throws org.apache.zookeeper.KeeperException

Throws:

org.apache.zookeeper.KeeperException

getRegionServerAccounting

public RegionServerAccounting getRegionServerAccounting()

Description copied from interface: RegionServerServices

Returns the RegionServerAccounting for this Region Server

Specified by:

getRegionServerAccounting in interface RegionServerServices

roundSize

private static int roundSize(long sizeInByte,
 int sizeUnit)

computeIfPersistentBucketCache

private void computeIfPersistentBucketCache(Consumer<BucketCache> computation)

createRegionLoad

org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad createRegionLoad(HRegion r,
 org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad.Builder regionLoadBldr,
 org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier.Builder regionSpecifier)
                                                                                           throws IOException

Parameters:

r - Region to get RegionLoad for.

regionLoadBldr - the RegionLoad.Builder, can be null

regionSpecifier - the RegionSpecifier.Builder, can be null

Returns:

RegionLoad instance.

Throws:

IOException

createUserLoad

private org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.UserLoad createUserLoad(String user,
 MetricsUserSource userSource)

createRegionLoad

public org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad createRegionLoad(String encodedRegionName)
                                                                                                  throws IOException

Throws:

IOException

isOnline

public boolean isOnline()

Report the status of the server. A server is online once all the startup is completed (setting up filesystem, starting executorService threads, etc.). This method is designed mostly to be useful in tests.

Returns:

true if online, false if not.

setupWALAndReplication

private void setupWALAndReplication()
                             throws IOException

Setup WAL log and replication if enabled. Replication setup is done in here because it wants to be hooked up to WAL.

Throws:

IOException

getWALEventTrackerListener

private WALActionsListener getWALEventTrackerListener(org.apache.hadoop.conf.Configuration conf)

startReplicationService

private void startReplicationService()
                              throws IOException

Start up replication source and sink handlers.

Throws:

IOException

getMasterAddressTracker

public MasterAddressTracker getMasterAddressTracker()

Returns Master address tracker instance.

startServices

private void startServices()
                    throws IOException

Start maintenance Threads, Server, Worker and lease checker threads. Start all threads we need to run. This is called after we've successfully registered with the Master. Install an UncaughtExceptionHandler that calls abort of RegionServer if we get an unhandled exception. We cannot set the handler on all threads. Server's internal Listener thread is off limits. For Server, if an OOME, it waits a while then retries. Meantime, a flush or a compaction that tries to run should trigger same critical condition and the shutdown will run. On its way out, this server will shut down Server. Leases are sort of inbetween. It has an internal thread that while it inherits from Chore, it keeps its own internal stop mechanism so needs to be stopped by this hosting server. Worker logs the exception and exits.

Throws:

IOException

initializeThreads

private void initializeThreads()

registerConfigurationObservers

private void registerConfigurationObservers()

isHealthy

private boolean isHealthy()

getWALs

public List<WAL> getWALs()

Description copied from interface: RegionServerServices

Returns the List of WALs that are used by this server Doesn't include the meta WAL

Specified by:

getWALs in interface RegionServerServices

getWAL

public WAL getWAL(RegionInfo regionInfo)
           throws IOException

Description copied from interface: RegionServerServices

Returns the WAL for a particular region. Pass null for getting the default (common) WAL

Specified by:

getWAL in interface RegionServerServices

Throws:

IOException

getWalRoller

public LogRoller getWalRoller()

getWalFactory

public WALFactory getWalFactory()

stop

public void stop(String msg)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Specified by:

stop in interface Stoppable

Parameters:

msg - Why we're stopping.

stop

public void stop(String msg,
 boolean force,
 User user)

Stops the regionserver.

Parameters:

msg - Status message

force - True if this is a regionserver abort

user - The user executing the stop request, or null if no user is associated

waitForServerOnline

public void waitForServerOnline()

postOpenDeployTasks

public void postOpenDeployTasks(RegionServerServices.PostOpenDeployContext context)
                         throws IOException

Description copied from interface: RegionServerServices

Tasks to perform after region open to complete deploy of region on regionserver

Specified by:

postOpenDeployTasks in interface RegionServerServices

Parameters:

context - the context

Throws:

IOException

skipReportingTransition

private boolean skipReportingTransition(RegionServerServices.RegionStateTransitionContext context)

Helper method for use in tests. Skip the region transition report when there's no master around to receive it.

createReportRegionStateTransitionRequest

private org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportRegionStateTransitionRequest createReportRegionStateTransitionRequest(RegionServerServices.RegionStateTransitionContext context)

reportRegionStateTransition

public boolean reportRegionStateTransition(RegionServerServices.RegionStateTransitionContext context)

Description copied from interface: RegionServerServices

Notify master that a handler requests to change a region state

Specified by:

reportRegionStateTransition in interface RegionServerServices

triggerFlushInPrimaryRegion

private void triggerFlushInPrimaryRegion(HRegion region)

Trigger a flush in the primary region replica if this region is a secondary replica. Does not block this thread. See RegionReplicaFlushHandler for details.

getRSRpcServices

@Private
public RSRpcServices getRSRpcServices()

abort

public void abort(String reason,
 Throwable cause)

Cause the server to exit without closing the regions it is serving, the log it is using and without notifying the master. Used unit testing and on catastrophic events such as HDFS is yanked out from under hbase or we OOME. the reason we are aborting the exception that caused the abort, or null

Specified by:

abort in interface Abortable

Parameters:

reason - Why we're aborting.

cause - Throwable that caused abort. Can be null.

kill

@Private
protected void kill()

scheduleAbortTimer

private void scheduleAbortTimer()

stopServiceThreads

protected void stopServiceThreads()

Wait on all threads to finish. Presumption is that all closes and stops have already been called.

getReplicationSourceService

public ReplicationSourceService getReplicationSourceService()

Returns Return the object that implements the replication source executorService.

Specified by:

getReplicationSourceService in interface RegionServerServices

getReplicationSinkService

public ReplicationSinkService getReplicationSinkService()

Returns Return the object that implements the replication sink executorService.

createRegionServerStatusStub

private ServerName createRegionServerStatusStub()

Get the current master from ZooKeeper and open the RPC connection to it. To get a fresh connection, the current rssStub must be null. Method will block until a master is available. You can break from this block by requesting the server stop.

Returns:

master + port, or null if server has been stopped

createRegionServerStatusStub

@Private
protected ServerName createRegionServerStatusStub(boolean refresh)

Get the current master from ZooKeeper and open the RPC connection to it. To get a fresh connection, the current rssStub must be null. Method will block until a master is available. You can break from this block by requesting the server stop.

Parameters:

refresh - If true then master address will be read from ZK, otherwise use cached data

Returns:

master + port, or null if server has been stopped

keepLooping

private boolean keepLooping()

Returns:

True if we should break loop because cluster is going down or this server has been stopped or hdfs has gone bad.

reportForDuty

private org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStartupResponse reportForDuty()
                                                                                                                      throws IOException

Throws:

IOException

getLastSequenceId

public org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionStoreSequenceIds getLastSequenceId(byte[] encodedRegionName)

Specified by:

getLastSequenceId in interface LastSequenceId

Parameters:

encodedRegionName - Encoded region name

Returns:

Last flushed sequence Id for region and its stores. Id will be -1 if it can't be determined

closeMetaTableRegions

private void closeMetaTableRegions(boolean abort)

Close meta region if we carry it

Parameters:

abort - Whether we're running an abort.

closeUserRegions

private void closeUserRegions(boolean abort)

Schedule closes on all user regions. Should be safe calling multiple times because it wont' close regions that are already closed or that are closing.

Parameters:

abort - Whether we're running an abort.

getOnlineRegions

protected Map<String,HRegion> getOnlineRegions()

getNumberOfOnlineRegions

public int getNumberOfOnlineRegions()

getOnlineRegionsLocalContext

public Collection<HRegion> getOnlineRegionsLocalContext()

For tests, web ui and metrics. This method will only work if HRegionServer is in the same JVM as client; HRegion cannot be serialized to cross an rpc.

addRegion

public void addRegion(HRegion region)

Description copied from interface: MutableOnlineRegions

Add to online regions.

Specified by:

addRegion in interface MutableOnlineRegions

addRegion

private void addRegion(SortedMap<Long,Collection<HRegion>> sortedRegions,
 HRegion region,
 long size)

getCopyOfOnlineRegionsSortedByOffHeapSize

SortedMap<Long,Collection<HRegion>> getCopyOfOnlineRegionsSortedByOffHeapSize()

Returns:

A new Map of online regions sorted by region off-heap size with the first entry being the biggest.

getCopyOfOnlineRegionsSortedByOnHeapSize

SortedMap<Long,Collection<HRegion>> getCopyOfOnlineRegionsSortedByOnHeapSize()

Returns:

A new Map of online regions sorted by region heap size with the first entry being the biggest.

getFlushRequester

public FlushRequester getFlushRequester()

Returns reference to FlushRequester

Specified by:

getFlushRequester in interface RegionServerServices

Returns:

Implementation of FlushRequester or null. Usually it will not be null unless during intialization.

getCompactionRequestor

public CompactionRequester getCompactionRequestor()

Specified by:

getCompactionRequestor in interface RegionServerServices

Returns:

Implementation of CompactionRequester or null. Usually it will not be null unless during intialization.

getLeaseManager

public LeaseManager getLeaseManager()

Description copied from interface: RegionServerServices

Returns The RegionServer's "Leases" service

Specified by:

getLeaseManager in interface RegionServerServices

isDataFileSystemOk

boolean isDataFileSystemOk()

Returns true when the data file system is available, false otherwise.

getRegionServerCoprocessorHost

public RegionServerCoprocessorHost getRegionServerCoprocessorHost()

getRegionsInTransitionInRS

public ConcurrentMap<byte[],Boolean> getRegionsInTransitionInRS()

Description copied from interface: RegionServerServices

Get the regions that are currently being opened or closed in the RS

Specified by:

getRegionsInTransitionInRS in interface RegionServerServices

Returns:

map of regions in transition in this RS

getRegionServerRpcQuotaManager

public RegionServerRpcQuotaManager getRegionServerRpcQuotaManager()

Description copied from interface: RegionServerServices

Returns RegionServer's instance of RegionServerRpcQuotaManager

Specified by:

getRegionServerRpcQuotaManager in interface RegionServerServices

createNewReplicationInstance

private static void createNewReplicationInstance(org.apache.hadoop.conf.Configuration conf,
 HRegionServer server,
 org.apache.hadoop.fs.FileSystem walFs,
 org.apache.hadoop.fs.Path walDir,
 org.apache.hadoop.fs.Path oldWALDir,
 WALFactory walFactory)
                                          throws IOException

Load the replication executorService objects, if any

Throws:

IOException

newReplicationInstance

private static <T extends ReplicationService> T newReplicationInstance(String classname,
 Class<T> xface,
 org.apache.hadoop.conf.Configuration conf,
 HRegionServer server,
 org.apache.hadoop.fs.FileSystem walFs,
 org.apache.hadoop.fs.Path logDir,
 org.apache.hadoop.fs.Path oldLogDir,
 WALFactory walFactory)
                                                                throws IOException

Throws:

IOException

getWalGroupsReplicationStatus

public Map<String,ReplicationStatus> getWalGroupsReplicationStatus()

constructRegionServer

static HRegionServer constructRegionServer(Class<? extends HRegionServer> regionServerClass,
 org.apache.hadoop.conf.Configuration conf)

Utility for constructing an instance of the passed HRegionServer class.

main

public static void main(String[] args)

See Also:

• HRegionServerCommandLine

getRegions

public List<HRegion> getRegions(TableName tableName)

Gets the online regions of the specified table. This method looks at the in-memory onlineRegions. It does not go to hbase:meta. Only returns online regions. If a region on this table has been closed during a disable, etc., it will not be included in the returned list. So, the returned list may not necessarily be ALL regions in this table, its all the ONLINE regions in the table.

Specified by:

getRegions in interface OnlineRegions

Specified by:

getRegions in interface RegionServerServices

Parameters:

tableName - table to limit the scope of the query

Returns:

Online regions from tableName

getRegions

public List<HRegion> getRegions()

Description copied from interface: OnlineRegions

Get all online regions in this RS.

Specified by:

getRegions in interface OnlineRegions

Specified by:

getRegions in interface RegionServerServices

Returns:

List of online Region

getOnlineTables

public Set<TableName> getOnlineTables()

Gets the online tables in this RS. This method looks at the in-memory onlineRegions.

Returns:

all the online tables in this RS

getRegionServerCoprocessors

public String[] getRegionServerCoprocessors()

closeRegionIgnoreErrors

private void closeRegionIgnoreErrors(RegionInfo region,
 boolean abort)

Try to close the region, logs a warning on failure but continues.

Parameters:

region - Region to close

closeRegion

protected boolean closeRegion(String encodedName,
 boolean abort,
 ServerName destination)
                       throws NotServingRegionException

Close asynchronously a region, can be called from the master or internally by the regionserver when stopping. If called from the master, the region will update the status.

If an opening was in progress, this method will cancel it, but will not start a new close. The coprocessors are not called in this case. A NotServingRegionException exception is thrown.

If a close was in progress, this new request will be ignored, and an exception thrown.

Provides additional flag to indicate if this region blocks should be evicted from the cache.

Parameters:

encodedName - Region to close

abort - True if we are aborting

destination - Where the Region is being moved too... maybe null if unknown.

Returns:

True if closed a region.

Throws:

NotServingRegionException - if the region is not online

getOnlineRegion

public HRegion getOnlineRegion(byte[] regionName)

Returns:

HRegion for the passed binary regionName or null if named region is not member of the online regions.

getRegion

public HRegion getRegion(String encodedRegionName)

Description copied from interface: OnlineRegions

Return Region instance. Only works if caller is in same context, in same JVM. Region is not serializable.

Specified by:

getRegion in interface OnlineRegions

Specified by:

getRegion in interface RegionServerServices

Returns:

Region for the passed encoded encodedRegionName or null if named region is not member of the online regions.

removeRegion

public boolean removeRegion(HRegion r,
 ServerName destination)

Description copied from interface: MutableOnlineRegions

Removes the given Region from the list of onlineRegions.

Specified by:

removeRegion in interface MutableOnlineRegions

Parameters:

r - Region to remove.

destination - Destination, if any, null otherwise.

Returns:

True if we removed a region from online list.

getRegion

protected HRegion getRegion(byte[] regionName)
                     throws NotServingRegionException

Protected Utility method for safely obtaining an HRegion handle.

Parameters:

regionName - Name of online HRegion to return

Returns:

HRegion for regionName

Throws:

NotServingRegionException

getRegionByEncodedName

public HRegion getRegionByEncodedName(String encodedRegionName)
                               throws NotServingRegionException

Throws:

NotServingRegionException

getRegionByEncodedName

private HRegion getRegionByEncodedName(byte[] regionName,
 String encodedRegionName)
                                throws NotServingRegionException

Throws:

NotServingRegionException

cleanup

private Throwable cleanup(Throwable t,
 String msg)

Cleanup after Throwable caught invoking method. Converts t to IOE if it isn't already.

Parameters:

t - Throwable

msg - Message to log in error. Can be null.

Returns:

Throwable converted to an IOE; methods can only let out IOEs.

convertThrowableToIOE

private IOException convertThrowableToIOE(Throwable t,
 String msg)

Parameters:

msg - Message to put in new IOE if passed t is not an IOE

Returns:

Make t an IOE if it isn't already.

checkFileSystem

boolean checkFileSystem()

Checks to see if the file system is still accessible. If not, sets abortRequested and stopRequested

Returns:

false if file system is not available

updateRegionFavoredNodesMapping

public void updateRegionFavoredNodesMapping(String encodedRegionName,
 List<org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ServerName> favoredNodes)

Description copied from interface: FavoredNodesForRegion

Used to update the favored nodes mapping when required.

Specified by:

updateRegionFavoredNodesMapping in interface FavoredNodesForRegion

getFavoredNodesForRegion

public InetSocketAddress[] getFavoredNodesForRegion(String encodedRegionName)

Return the favored nodes for a region given its encoded name. Look at the comment around regionFavoredNodesMap on why we convert to InetSocketAddress[] here.

Specified by:

getFavoredNodesForRegion in interface FavoredNodesForRegion

Parameters:

encodedRegionName - the encoded region name.

Returns:

array of favored locations

getNonceManager

public ServerNonceManager getNonceManager()

Description copied from interface: RegionServerServices

Only required for "old" log replay; if it's removed, remove this.

Specified by:

getNonceManager in interface RegionServerServices

Returns:

The RegionServer's NonceManager

addToMovedRegions

private void addToMovedRegions(String encodedName,
 ServerName destination,
 long closeSeqNum,
 boolean selfMove)

removeFromMovedRegions

void removeFromMovedRegions(String encodedName)

getMovedRegion

@Private
public HRegionServer.MovedRegionInfo getMovedRegion(String encodedRegionName)

movedRegionCacheExpiredTime

@Private
public int movedRegionCacheExpiredTime()

getMyEphemeralNodePath

private String getMyEphemeralNodePath()

isHealthCheckerConfigured

private boolean isHealthCheckerConfigured()

getCompactSplitThread

public CompactSplit getCompactSplitThread()

Returns the underlying CompactSplit for the servers

execRegionServerService

org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.CoprocessorServiceResponse execRegionServerService(org.apache.hbase.thirdparty.com.google.protobuf.RpcController controller,
 org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.CoprocessorServiceRequest serviceRequest)
                                                                                                           throws org.apache.hbase.thirdparty.com.google.protobuf.ServiceException

Throws:

org.apache.hbase.thirdparty.com.google.protobuf.ServiceException

getBlockCache

public Optional<BlockCache> getBlockCache()

May be null if this is a master which not carry table.

Specified by:

getBlockCache in interface RegionServerServices

Returns:

The block cache instance used by the regionserver.

getMobFileCache

public Optional<MobFileCache> getMobFileCache()

May be null if this is a master which not carry table.

Specified by:

getMobFileCache in interface RegionServerServices

Returns:

The cache for mob files used by the regionserver.

clearRegionBlockCache

CacheEvictionStats clearRegionBlockCache(Region region)

getCompactionPressure

public double getCompactionPressure()

Specified by:

getCompactionPressure in interface RegionServerServices

Returns:

the max compaction pressure of all stores on this regionserver. The value should be greater than or equal to 0.0, and any value greater than 1.0 means we enter the emergency state that some stores have too many store files.

See Also:

• Store.getCompactionPressure()

getHeapMemoryManager

public HeapMemoryManager getHeapMemoryManager()

Description copied from interface: RegionServerServices

Returns heap memory manager instance

Specified by:

getHeapMemoryManager in interface RegionServerServices

getMemStoreFlusher

public MemStoreFlusher getMemStoreFlusher()

walRollRequestFinished

@Private
public boolean walRollRequestFinished()

For testing

Returns:

whether all wal roll request finished for this regionserver

getFlushThroughputController

public ThroughputController getFlushThroughputController()

Description copied from interface: RegionServerServices

Returns the controller to avoid flush too fast

Specified by:

getFlushThroughputController in interface RegionServerServices

getFlushPressure

public double getFlushPressure()

Specified by:

getFlushPressure in interface RegionServerServices

Returns:

the flush pressure of all stores on this regionserver. The value should be greater than or equal to 0.0, and any value greater than 1.0 means we enter the emergency state that global memstore size already exceeds lower limit.

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration newConf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

getMetrics

public MetricsRegionServer getMetrics()

Description copied from interface: RegionServerServices

Returns the metrics tracker for the region server

Specified by:

getMetrics in interface RegionServerServices

getSecureBulkLoadManager

public SecureBulkLoadManager getSecureBulkLoadManager()

Description copied from interface: RegionServerServices

Returns RegionServer's instance of SecureBulkLoadManager

Specified by:

getSecureBulkLoadManager in interface RegionServerServices

regionLock

public EntityLock regionLock(List<RegionInfo> regionInfo,
 String description,
 Abortable abort)

Description copied from interface: RegionServerServices

Master based locks on namespaces/tables/regions.

Specified by:

regionLock in interface RegionServerServices

unassign

public void unassign(byte[] regionName)
              throws IOException

Description copied from interface: RegionServerServices

Unassign the given region from the current regionserver and assign it randomly. Could still be assigned to us. This is used to solve some tough problems for which you need to reset the state of a region. For example, if you hit FileNotFound exception and want to refresh the store file list.

See HBASE-17712 for more details.

Specified by:

unassign in interface RegionServerServices

Throws:

IOException

getRegionServerSpaceQuotaManager

public RegionServerSpaceQuotaManager getRegionServerSpaceQuotaManager()

Description copied from interface: RegionServerServices

Returns RegionServer's instance of RegionServerSpaceQuotaManager

Specified by:

getRegionServerSpaceQuotaManager in interface RegionServerServices

reportFileArchivalForQuotas

public boolean reportFileArchivalForQuotas(TableName tableName,
 Collection<Map.Entry<String,Long>> archivedFiles)

Description copied from interface: RegionServerServices

Reports a collection of files, and their sizes, that belonged to the given table were just moved to the archive directory.

Specified by:

reportFileArchivalForQuotas in interface RegionServerServices

Parameters:

tableName - The name of the table that files previously belonged to

archivedFiles - Files and their sizes that were moved to archive

Returns:

true if the files were successfully reported, false otherwise.

executeProcedure

void executeProcedure(long procId,
 long initiatingMasterActiveTime,
 RSProcedureCallable callable)

remoteProcedureComplete

public void remoteProcedureComplete(long procId,
 long initiatingMasterActiveTime,
 Throwable error)

reportProcedureDone

void reportProcedureDone(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.ReportProcedureDoneRequest request)
                  throws IOException

Throws:

IOException

submitRegionProcedure

boolean submitRegionProcedure(long procId)

Will ignore the open/close region procedures which already submitted or executed. When master had unfinished open/close region procedure and restarted, new active master may send duplicate open/close region request to regionserver. The open/close request is submitted to a thread pool and execute. So first need a cache for submitted open/close region procedures. After the open/close region request executed and report region transition succeed, cache it in executed region procedures cache. See finishRegionProcedure(long). After report region transition succeed, master will not send the open/close region request to regionserver again. And we thought that the ongoing duplicate open/close region request should not be delayed more than 600 seconds. So the executed region procedures cache will expire after 600 seconds. See HBASE-22404 for more details.

Parameters:

procId - the id of the open/close region procedure

Returns:

true if the procedure can be submitted.

finishRegionProcedure

public void finishRegionProcedure(long procId)

See submitRegionProcedure(long).

Parameters:

procId - the id of the open/close region procedure

getCompactedHFilesDischarger

@Private
public CompactedHFilesDischarger getCompactedHFilesDischarger()

getRetryPauseTime

@Private
public long getRetryPauseTime()

Return pause time configured in HConstants.HBASE_RPC_SHORTOPERATION_RETRY_PAUSE_TIME}

Returns:

pause time

getActiveMaster

public Optional<ServerName> getActiveMaster()

Description copied from interface: ConnectionRegistryEndpoint

Get active master address.

Specified by:

getActiveMaster in interface ConnectionRegistryEndpoint

getBackupMasters

public List<ServerName> getBackupMasters()

Description copied from interface: ConnectionRegistryEndpoint

Get backup masters address.

Specified by:

getBackupMasters in interface ConnectionRegistryEndpoint

getBootstrapNodes

public Iterator<ServerName> getBootstrapNodes()

Description copied from interface: ConnectionRegistryEndpoint

Get a iterator of the available bootstrap nodes.

Specified by:

getBootstrapNodes in interface ConnectionRegistryEndpoint

getMetaLocations

public List<HRegionLocation> getMetaLocations()

Description copied from interface: ConnectionRegistryEndpoint

Get the location of meta regions.

Specified by:

getMetaLocations in interface ConnectionRegistryEndpoint

createNamedQueueRecord

protected NamedQueueRecorder createNamedQueueRecord()

Specified by:

createNamedQueueRecord in class HBaseServerBase<RSRpcServices>

clusterMode

protected boolean clusterMode()

Specified by:

clusterMode in class HBaseServerBase<RSRpcServices>

getBrokenStoreFileCleaner

@Private
public BrokenStoreFileCleaner getBrokenStoreFileCleaner()

getRSMobFileCleanerChore

@Private
public RSMobFileCleanerChore getRSMobFileCleanerChore()

getRsSnapshotVerifier

RSSnapshotVerifier getRsSnapshotVerifier()

stopChores

protected void stopChores()

Specified by:

stopChores in class HBaseServerBase<RSRpcServices>

getRegionReplicationBufferManager

public RegionReplicationBufferManager getRegionReplicationBufferManager()

Specified by:

getRegionReplicationBufferManager in interface RegionServerServices